An intelligent automobile grading early warning system based on driver risk perception reliability

Abstract

The application relates to an intelligent automobile hierarchical early warning system based on driver risk perception reliability, which comprises a driver risk area judgment module based on eye movement information, an objective environment anisotropic risk field calculation module, an objective environment anisotropic risk field visual field conversion module, an intelligent automobile risk area judgment module based on driver intention, a human-vehicle risk perception result representation module based on risk perception accumulation effect and decay effect, a driver risk perception reliability quantification module and a hierarchical early warning module based on driver risk perception reliability; the application realizes longitudinal and lateral combined early warning from the driver risk perception level by constructing an anisotropic driving risk field, dividing a driving visual field area, capturing a driver visual attention point and performing space-time convolution operation, effectively solves problems such as inaccurate traffic environment risk description, difficult quantitative description and evaluation of driver risk perception conditions, longitudinal and lateral early warning fragmentation and early warning lag and the like, and improves the performance of the early warning system.

Description

Technical Field

[0001] This invention relates to an intelligent vehicle early warning system, and more particularly to an intelligent vehicle graded early warning system based on the reliability of driver risk perception. Background Technology

[0002] In recent years, automotive intelligent technology has developed rapidly. Compared with traditional cars, intelligent cars are equipped with more powerful environmental perception systems, thus offering greater advantages in ensuring driving safety. Currently, various safety warning systems have been mass-produced and applied as standard or optional features in some intelligent vehicle models. However, existing intelligent vehicle warning systems still face some pressing technical challenges that need to be addressed:

[0003] 1. The prerequisite for building any early warning system is an accurate description of traffic environment risks. Existing related technologies generally use vehicle distance, following distance, collision time and their derivative indicators to describe traffic environment risks. However, traffic participants in the traffic environment are diverse, dynamic and time-varying and their states are random. The above indicators currently used are difficult to accurately describe the objective risks of the traffic environment and their evolution trend, which can easily lead to early warning, delayed warning or even missed risk detection, directly affecting the performance of the early warning system and driving safety.

[0004] 2. Existing technologies generally calculate risk indicators based on the vehicle's motion state to generate warning signals. However, the vehicle's motion state is caused by the driver's driving behavior, which is essentially based on the driver's perception. Therefore, existing warning systems based on vehicle status all have an inherent time lag, posing a significant safety hazard in the rapidly changing real-world traffic environment. Intelligent vehicle warning systems based on driver risk perception can provide warnings at this fundamental level, even preventing dangerous driving behaviors before they occur, greatly improving driving safety. However, due to difficulties in quantifying driver risk perception and the lack of robust methods for assessing driver risk perception levels, no technology or product currently exists that provides warnings based on driver risk perception.

[0005] 3. In existing related technologies, intelligent vehicle warning systems typically target either longitudinal driving behavior (e.g., forward collision warning systems, graded automatic braking systems, etc.) or lateral driving behavior (e.g., lane departure warning systems, lane change assist systems, etc.). However, both driving risks and driver behavior have strong longitudinal and lateral coupling characteristics, and existing independent longitudinal and lateral warning systems lack effective coordination mechanisms, resulting in extremely limited performance of existing warning systems in real traffic environments.

[0006] Therefore, developing an intelligent vehicle early warning system that uses the driver's subjective risk perception level as the core criterion for early warning, can accurately describe the objective risks of the traffic environment, and can simultaneously achieve longitudinal and lateral coupled risk early warning has become a key issue that the entire industry urgently needs to solve. Summary of the Invention

[0007] To address the aforementioned technical problems, this invention provides an intelligent vehicle graded early warning system based on the reliability of driver risk perception, comprising: a driver risk area determination module based on eye-tracking information, an objective environment anisotropic risk field calculation module, an objective environment anisotropic risk field field view conversion module, an intelligent vehicle risk area determination module based on driver intent, a human-vehicle risk perception result characterization module based on risk perception cumulative effect and decay effect, a driver risk perception reliability quantification module, and a graded early warning module based on the reliability of driver risk perception.

[0008] (1) The driver risk area determination module based on eye movement information takes driver eye movement information as input and outputs the risk area determined by the driver according to the location of the driver's visual attention point.

[0009] (2) The objective environment anisotropic risk field calculation module takes traffic environment information as input and outputs anisotropic risk field in the world coordinate system;

[0010] (3) The objective environment anisotropic risk field field of view conversion module transforms the anisotropic risk field in the world coordinate system to the driver's field of view through coordinate transformation of the world coordinate system-camera coordinate system-image coordinate system-pixel coordinate system, and outputs the anisotropic risk field in the driver's field of view.

[0011] (4) The intelligent vehicle risk area determination module based on driver intention takes the anisotropic risk field under the driver's field of vision and the driver's driving intention as input, and outputs the risk area determined by the intelligent vehicle according to the magnitude of the risk field and the driver's driving intention; the driver's driving intention includes left lane change, lane keeping and right lane change.

[0012] (5) The human-vehicle risk perception result characterization module based on the cumulative effect and decay effect of risk perception takes the risk area determined by the driver and the risk area determined by the intelligent vehicle as input, and characterizes the cumulative effect of risk perception by statistically analyzing the human-vehicle risk perception area over a period of time, and characterizes the decay effect of risk perception by spatiotemporal convolution operation based on the semi-Hanning window function, thereby characterizing and outputting the risk perception results of the driver and the intelligent vehicle.

[0013] (6) The driver risk perception reliability quantification module takes the risk area determined by the driver, the risk area determined by the intelligent vehicle, and the risk perception results of the driver and the intelligent vehicle as input, and generates and outputs a driver risk perception reliability factor that can quantify the reliability of the driver's risk perception through region matching.

[0014] (7) The graded early warning module based on driver risk perception reliability sets three levels of thresholds and divides the early warning level into four levels according to the driver risk perception reliability factor, thus completing the intelligent vehicle graded early warning starting from the driver risk perception layer.

[0015] Furthermore, the driver's field of vision is divided into three main directions: forward, left (left rearview mirror), and right (right rearview mirror). The driver's forward field of vision is further divided into six regions: upper left, upper, upper right, lower left, lower, and lower right, numbered 1-6 respectively. The left and right fields of vision are numbered 7 and 8 respectively.

[0016] Furthermore, the driver risk area determination module based on eye-tracking information takes driver eye-tracking information as input. The driver's eye-tracking behavior includes fixation behavior and saccade behavior. The eye-tracking device collects the driver's eye-tracking information and captures the driver's fixation behavior from it, thereby obtaining the area where the fixation point is located. When the driver fixates on a certain area, that area is the risk area determined by the driver. The driver risk area determination module based on eye-tracking information determines from the driver's eye-tracking information collected by the eye-tracking device that the driver's fixation point falls on one or more areas of his / her field of vision, thereby obtaining the corresponding number of the risk area determined by the driver.

[0017] Furthermore, the objective environment anisotropic risk field calculation module receives traffic environment information and, based on the basic types, geometric shapes, defined attributes, and motion states of traffic participants, describes the objective risks they generate by establishing an anisotropic risk field. This anisotropic risk field is divided into a dynamic anisotropic risk field describing moving vehicles, a static anisotropic risk field describing static obstacles, and a lane line filtering effect.

[0018] (2.1) The anisotropy of the dynamic anisotropic risk field is reflected in the motion state of the vehicle.

[0019] In the world coordinate system, the vehicle K(x) K ,y K ) and a point j(x) in the field j ,y j The distance vector d between ) K,j for:

[0020] d K,j =(x j-x K ,y j -y K (1)

[0021] Establish a coordinate system X fixed on K K O K Y K O K Let K be the center of mass, OX K OY represents the direction of K's movement. K The normal vector representing the direction of K's movement; d in the world coordinate system K,j Convert to X K O K Y K Equivalent distance vector in coordinate system

[0022]

[0023] in, and OX in world coordinate system K and OY K The unit vector, α K and β K OX K and OY K The distance scaling factor in the direction is related to the vehicle body geometry of K; therefore, the dynamic anisotropic risk field E generated by K... K Defined as:

[0024]

[0025] Among them, E K The direction is from the center of K to point j. and Let K be the acceleration in the forward direction and the acceleration in the normal direction, respectively. and Let μ be the acceleration coefficient, and μ0 be the peak field strength occurring at the center of K. and They are respectively The angle between M and the direction of travel of K and the normal, K To account for the equivalent mass of the vehicle type and speed:

[0026]

[0027] Where, m K For the true mass of K, v K Let b be the velocity in the forward direction of K. m and k m A constant related to the type of K;

[0028] Furthermore, when vehicle K does not intend to change lanes, the electric field strength E generated by K in the adjacent lane is... K It needs to be multiplied by the lane line filtering factor f L ;

[0029] (2.2) The lane line filtering effect is related to the lane line type and the vehicle's motion intention. When the vehicle's intention is to maintain its original lane, the lane line filtering factor f is defined. L :

[0030]

[0031] Among them, T L d is the lane line type coefficient. L,K Let d be the distance d is the distance between the edge of the vehicle K and the lane line. L,center K represents the distance between the edge of the vehicle body and the lane line when the vehicle is traveling along the center line of the current lane; however, when the vehicle intends to change lanes, the lane boundary between the original lane and the target lane will lose its filtering effect on the dynamic anisotropic risk field generated by the vehicle.

[0032] (2.3) The static anisotropic risk field is used to describe the risk generated by static obstacles in the traffic environment, and a coordinate system X is established fixed on the static element S. S O S Y S O S Let S be the centroid, and OX be the centroid. S OY represents the direction of the major axis of the circumscribed ellipse S. S Represents the direction of the minor axis of the circumscribed ellipse S; the distance vector d in the world coordinate system S,j Convert to X S O S Y S Equivalent distance vector in coordinate system

[0033]

[0034] in, and OX in world coordinate system S and OY S The unit vector, α S and β S OX S and OY S The distance scaling factor in the direction is related to the geometric dimensions of S; referring to equation (3), the static anisotropic risk field E generated by S S Defined as:

[0035]

[0036] Where, m S For the true mass of S, μ S The peak value of the static anisotropic risk field appearing at the S-center of mass;

[0037] In a certain traffic environment, the total field strength E of the objective environment anisotropic risk field obtained by the objective environment anisotropic risk field calculation module is expressed as:

[0038]

[0039] Wherein, M and N are the number of vehicles and static obstacles in the traffic environment that affect the driving safety of the vehicle, respectively; the objective environment anisotropic risk field calculation module selects the longitudinal range of traffic participants as 150m in front of and behind the vehicle, and the lateral range as the vehicle's current lane and the adjacent lanes to the left and right of the current lane.

[0040] Furthermore, the objective environment anisotropic risk field field of view conversion module transforms the anisotropic risk field in the world coordinate system to the driver's field of view through coordinate transformation from the world coordinate system to the camera coordinate system to the image coordinate system to the pixel coordinate system, and outputs the anisotropic risk field in the driver's field of view.

[0041] Among them, the dividing line is drawn according to the position of the front seat of the vehicle. The risk field in the left and right rear areas of the dividing line does not need to be transformed into a coordinate system. The risk field in front of the dividing line is projected onto the driver's forward vision and forms a matching relationship with the 6 areas in the driver's forward vision.

[0042] The objective environment anisotropic risk field view transformation module defines four coordinate systems: world coordinate system, camera coordinate system, image coordinate system, and pixel coordinate system, wherein:

[0043] The world coordinate system is a three-dimensional coordinate system that describes the position of an object in the real physical world, with the X-axis as the coordinate axis. w Y w and Z w The coordinate axes are perpendicular to each other, with units of meters (m). The camera coordinate system is a three-dimensional coordinate system established on the camera, with the camera's optical center O. c With the origin as the coordinate axis, X c Y c and Z c perpendicular to each other, X c and Y c X parallel to the image formed by the camera i axis and Y i Axial direction, Z c Parallel to the camera's principal optical axis, with coordinate axes in meters (m); the image coordinate system is a two-dimensional coordinate system describing the image captured by the camera, with its origin O. iThe intersection of the principal optical axis and the image plane, coordinate axis X i and Y i The axes are perpendicular to each other, and the unit of measurement is mm. The pixel coordinate system is a two-dimensional coordinate system that describes the position of a pixel on the image captured by the camera, with the origin at the image vertex O. UV The coordinate axes U and V are parallel to the X-axis of the image coordinate system. i axis and Y i Axis, the unit of coordinate axes is pixel;

[0044] Let the coordinates of a point P in the world coordinate system be (x... w ,y w ,z w If the coordinates are such that the transformation to the pixel coordinate system requires the following process:

[0045] (3.1) Transformation from world coordinate system to camera coordinate system: First, the origin of the world coordinate system is translated to the origin of the camera coordinate system through a translation transformation. Then, the camera coordinate system is rotated around the three axes of the world coordinate system. The transformation from world coordinate system to camera coordinate system is as follows:

[0046]

[0047] Among them, (x c ,y c ,z c (t1, t2, t3) are the coordinates of point P in the camera coordinate system, (t1, t2, t3) are the positions of the origin of the camera coordinate system in the world coordinate system, and R is the rotation matrix; further transformation of equation (9) yields the homogeneous matrix form:

[0048]

[0049] Where T is the translation matrix, C Int The extrinsic parameter matrix of the camera;

[0050] (3.2) Transformation from camera coordinate system to image coordinate system: The 3D coordinates are projected to 2D using perspective photography relationships. The transformation relationship is as follows:

[0051]

[0052] Among them, (x i ,y i ,1) are the normalized coordinates of point P in the image coordinate system, and f is the camera focal length;

[0053] (3.3) Transformation from image coordinate system to pixel coordinate system: The transformation relationship is as follows:

[0054]

[0055] Where (u,v,1) are the normalized coordinates of point P in the pixel coordinate system;

[0056] Combining equations (9) to (12), the transformation relationship from the world coordinate system to the pixel coordinate system is obtained as follows:

[0057]

[0058]

[0059] Among them, C Ext This is the camera extrinsic parameter matrix;

[0060] By transforming the world coordinate system to the pixel coordinate system as shown in equation (13), the anisotropic risk field of the objective environment is transformed into the driver's field of vision.

[0061] In order to establish a one-to-one correspondence with the driver's eight visual field areas, the anisotropic risk field of the objective environment under the driver's field of vision is divided into eight areas that are the same as the driver's field of vision. Area 7 and Area 8 represent the driver's left field of vision (left rearview mirror) and right field of vision (right rearview mirror), respectively.

[0062] Furthermore, the intelligent vehicle risk area determination module based on driver intent takes the anisotropic risk field under the driver's field of vision and the driver's driving intent as input:

[0063] (4.1) First, the intelligent vehicle risk area determination module based on driver intention selects the area that affects the driver's driving intention from 8 areas according to the driver's driving intention;

[0064] (4.2) In areas that affect the driver’s driving intention, if there are grid points in a certain area where the total field strength E of the objective environment anisotropic risk field is greater than 0.5, then the intelligent vehicle risk area determination module based on the driver’s driving intention will take that area as the risk area determined by the intelligent vehicle.

[0065] Furthermore, the human-vehicle risk perception result representation module based on the cumulative and decay effects of risk perception takes the risk areas determined by the driver and the intelligent vehicle as inputs, and quantifies the risk perception results of the driver and the intelligent vehicle respectively. When describing the risk perception results of the driver and the intelligent vehicle, the human-vehicle risk perception result representation module based on the cumulative effect of risk over the past 4 seconds takes into account. At the current time t0, the driver's risk perception result is related to all areas that the driver has looked at from t0-4 to t0. The decay effect of risk perception refers to the fact that after the driver looks at a certain area, the level of risk perception in that area will gradually decay over time.

[0066] Define dr k (t) is used to mark whether the region k at sampling time t is the effective region of the driver's risk perception, dr k (t) = 1 means that dr k (t) = 0 indicates no; the human-vehicle risk perception result characterization module based on the cumulative and decay effects of risk perception will dr k Treating the input signal as a step, the cumulative effect of driver risk perception is described by convolution operations based on a sliding time window, using a semi-Hanning window function H. h To describe the decay effect of driver risk perception, we use dr k and H h The convolution operation yields the driver risk perception result DR, which takes into account both cumulative and decay effects. k :

[0067]

[0068] Similarly, the human-vehicle risk perception result characterization module based on the cumulative and decay effects of risk perception applies the cumulative and decay effects of risk perception to the risk perception result IR of intelligent vehicles. k Construction:

[0069]

[0070] Among them, ir k (t) is used to mark whether region k at sampling time t is a risk region determined by the intelligent vehicle, ir k (t) = 1 means that ir k (t) = 0 means it is not.

[0071] Furthermore, the driver risk perception reliability quantification module comprehensively calculates and outputs the driver risk perception reliability factor (RPRF) based on the risk areas determined by the driver, the risk areas determined by the intelligent vehicle, the driver's risk perception results, and the intelligent vehicle's risk perception results. This quantitatively describes the reliability of the driver's risk perception. The specific calculation process of RPRF is as follows:

[0072] (6.1) Matching of risk areas determined by the driver and the intelligent vehicle: The driver risk perception reliability quantification module divides the matching degree r between the risk areas determined by the driver and the risk areas determined by the intelligent vehicle into 5 levels: high (r=1), medium-high (r=0.8), medium (r=0.5), medium-low (r=0.3), and low (r=0). When the intelligent vehicle determines multiple areas as risk areas at the same time, r takes the maximum value among them. When the driver's gaze point is not captured in any of the 8 areas, r for all areas is 0.

[0073] (6.2) Driver Risk Perception Reliability Factor (RPRF) Calculation: The Driver Risk Perception Reliability Quantification Module defines the Driver Risk Perception Reliability Factor (RPRF) to quantitatively describe the driver's perceived risk reliability. The calculation of RPRF includes three cases:

[0074] ①If This indicates that the driver's gaze points within the past 4 seconds are all outside the 8 areas of the driver's field of vision or the gaze points cannot be detected. It is considered that the driver's attention is distracted or unable to effectively perceive environmental risks. At this time, the driver's risk perception reliability is the lowest. The driver risk perception reliability quantification module sets RPRF=0.

[0075] ②If This indicates that the intelligent vehicle perceives no risk in the environment. This indicates that the driver has perceived a risk that the intelligent vehicle has not perceived. At this time, the driver's risk perception reliability is the highest, and the driver risk perception reliability quantification module sets RPRF=1.

[0076] ③If and This indicates that both the driver and the intelligent vehicle have effectively perceived the risks in the environment. Therefore, the driver risk perception reliability quantification module needs to calculate the driver risk perception reliability based on the risk perception results of the driver and the intelligent vehicle. The driver risk perception reliability quantification module sets two parameters, δ1 and δ2, for the calculation of the driver risk perception reliability factor RPRF.

[0077] The δ1 is used to characterize the driver's risk perception in the three main driver visual directions: forward, left, and right.

[0078]

[0079] in, and The human-vehicle risk perception matching degree is represented by the driver's forward, left and right vision respectively, and the value of δ1 is 0 or 1.

[0080] Based on using δ1 to characterize the driver's risk perception in the three main directions of the driver's field of vision, the driver risk perception reliability quantification module sets δ2 to characterize the driver's risk perception in eight specific areas of the driver's field of vision:

[0081]

[0082] Where N is the number of sampling points contained in [t0-4, t0]; the value range of δ2 is [0, 1].

[0083] Based on δ1 and δ2, the driver risk perception reliability quantification module calculates the driver risk perception reliability factor RPRF using the following formula:

[0084] RPRF=δ1·δ2 (19)

[0085] The value range of RPRF is [0, 1].

[0086] Furthermore, the graded early warning module based on driver risk perception reliability uses the driver's risk perception reliability factor (RPRF) as a basis and issues early warnings at four levels. The module has three early warning thresholds: RPRF1, RPRF2, and RPRF3, with the following values:

[0087]

[0088] The four levels of warning methods and activation ranges in the graded warning module based on the reliability of driver risk perception are as follows:

[0089] (7.1) When RPRF > RPRF1, it indicates that the driver's risk perception reliability is extremely high, and the driver can even perceive risks that the intelligent vehicle does not perceive. Therefore, the driver's driving behavior is considered reliable, and the graded warning module based on the reliability of driver risk perception does not need to issue a warning.

[0090] (7.2) When RPRF2<RPRF≤RPRF1, it indicates that the driver’s risk perception reliability has decreased. It is believed that the driver has not fully perceived the environmental risks and is prone to slightly risky driving behavior. At this time, the graded warning module based on the driver’s risk perception reliability will guide the driver to pay attention to one or several risk areas that were not previously paid attention to through visual prompts, so as to correct the driving behavior in time.

[0091] (7.3) When RPRF3<RPRF≤RPRF2, it indicates that the driver’s risk perception reliability has been reduced to a certain level, and the driver is very likely to engage in dangerous driving behavior. At this time, the graded warning module based on the driver’s risk perception reliability will warn the driver through more intense tactile cues.

[0092] (7.4) When RPRF≤RPRF3, it means that the driver’s risk perception reliability is extremely low and the driver’s driving behavior is carried out without effectively perceiving the environmental risks. The driver’s driving behavior is considered extremely dangerous. At this time, the graded early warning module based on the driver’s risk perception reliability will perform emergency avoidance.

[0093] The beneficial effects of this invention are:

[0094] The intelligent vehicle graded early warning system based on the reliability of driver risk perception described in this invention describes the risks generated by other traffic participants in the traffic environment by constructing an anisotropic risk field. It fully considers the impact of the basic types, geometric shapes, defined attributes, and motion states of traffic participants on the risks they generate, effectively solving the problem of inaccurate description of traffic environment risks in the prior art and improving the success rate of the intelligent vehicle early warning system.

[0095] This invention acquires the risk areas determined by the driver through techniques such as dividing the driver's field of vision and capturing the driver's visual attention points based on eye-tracking information; it acquires the risk areas determined by the intelligent vehicle through techniques such as projecting the driving risk field onto the driver's field of vision plane and determining the risk areas based on the driver's driving intention; it quantifies and characterizes the risk perception results of the driver and the intelligent vehicle through techniques such as historical risk assessment area statistics and spatiotemporal convolution operations; and it acquires the driver's risk perception reliability factor through techniques such as region matching to quantify and describe the driver's risk perception situation, and uses it as the basis for graded early warning in the system described in this invention. Thus, this invention can provide early warning to the driver from the fundamental level of perception, and even nip dangerous driving behaviors in the bud before they occur, effectively solving the core technical problems of difficulty in quantifying and evaluating the driver's risk perception situation and the lag in early warning in the prior art.

[0096] This invention, through technical means such as constructing an anisotropic driving risk field and introducing driver intent, fully considers the longitudinal and lateral coupling characteristics of driving risks and driver behavior, realizes longitudinal and lateral joint early warning, improves the performance of the early warning system in actual traffic environment, and effectively solves the problem of the separation of longitudinal and lateral early warning in the prior art. Attached Figure Description

[0097] Figure 1 This is a schematic diagram of the overall architecture of the present invention.

[0098] Figure 2 This is a schematic diagram of the driver's field of vision zoning as described in this invention.

[0099] Figure 3 This is a schematic diagram illustrating the effect of the dynamic anisotropic risk field of the present invention.

[0100] Figure 4 This is a schematic diagram illustrating the effect of the static anisotropic risk field of the present invention.

[0101] Figure 5 This is a schematic diagram illustrating the field of view conversion range and effect of the anisotropic risk field of the present invention.

[0102] Figure 6 This is a schematic diagram illustrating the matching rules between the risk areas determined by the driver and the intelligent vehicle in this invention.

[0103] Figure 7 This is a schematic diagram of the graded early warning rules based on the reliability of driver risk perception in this invention. Detailed Implementation

[0104] This invention provides an intelligent vehicle graded early warning system based on the reliability of driver risk perception, comprising: a driver risk area determination module based on eye-tracking information, an objective environment anisotropic risk field calculation module, an objective environment anisotropic risk field field view conversion module, an intelligent vehicle risk area determination module based on driver intention, a human-vehicle risk perception result characterization module based on risk perception cumulative effect and decay effect, a driver risk perception reliability quantification module, and a graded early warning module based on the reliability of driver risk perception.

[0105] (1) The driver risk area determination module based on eye movement information takes the driver's eye movement information as input and outputs the risk area determined by the driver according to the location of the driver's visual attention point:

[0106] See appendix Figure 1 The driver risk area determination module based on eye-tracking information takes driver eye-tracking information as input. Driver eye-tracking behavior includes three types: fixation, saccades, and smooth tracking. Fixation and smooth tracking obtain effective visual information through observation, while saccades involve extremely rapid eye movements, during which almost no effective visual information is obtained. Therefore, this invention categorizes fixation and smooth tracking behavior together as driver fixation behavior. When a driver fixates on a certain area, it can be considered that the driver can fully observe that area, and that area is the risk area determined by the driver. The driver risk area determination module based on eye-tracking information collects driver eye-tracking information using an eye tracker and captures the driver's fixation behavior from it, thereby obtaining the area where the fixation point is located.

[0107] See appendix Figure 2(a) This invention divides the driver's field of vision into three main directions: forward, left (left rearview mirror), and right (right rearview mirror). See appendix. Figure 2 (b) The present invention further divides the driver's forward visual field into six regions: upper left, upper, upper right, lower left, lower, and lower right. These six regions are designated as regions 1-6, and the left and right visual fields are designated as regions 7 and 8, respectively. The driver risk region determination module based on eye-tracking information can determine which region(s) the driver's gaze falls on within their visual field from the eye-tracking information collected by the eye tracker, thereby obtaining the identification number of the risk region determined by the driver.

[0108] (2) The objective environment anisotropic risk field calculation module takes traffic environment information as input and outputs anisotropic risk field in the world coordinate system:

[0109] See appendix Figure 1 The objective environment anisotropic risk field calculation module receives traffic environment information and, based on the basic types, geometric shapes, defined attributes, and motion states of traffic participants, describes the objective risks they generate by establishing an anisotropic risk field. This anisotropic risk field is divided into a dynamic anisotropic risk field describing moving vehicles and a static anisotropic risk field describing static obstacles. Furthermore, the anisotropic risk field also considers the filtering effect of lane markings on the risks posed to traffic participants.

[0110] (2.1) The anisotropy of the dynamic anisotropic risk field is reflected in the motion state of the vehicle. Current technologies consider the impact of the vehicle's longitudinal acceleration on the risk field, constructing a risk field with an asymmetric distribution in the longitudinal direction. The dynamic anisotropic risk field constructed in this invention further considers the impact of the vehicle's lateral motion on the risk, such as turning and lane changing.

[0111] In the world coordinate system, the vehicle K(x) K ,y K ) and a point j(x) in the field j ,y j The distance vector d between ) K,j for:

[0112] d K,j =(x j -x K ,y j -y K (1)

[0113] Establish a coordinate system X fixed on K K O K Y K OK Let K be the center of mass, OX K OY represents the direction of K's movement. K The normal vector represents the direction of K's movement. The d vector in the world coordinate system... K,j Convert to X K O K Y K Equivalent distance vector in coordinate system

[0114]

[0115] in, and OX in world coordinate system K and OY K The unit vector, α K and β K OX K and OY K The distance scaling factor in the direction is related to the vehicle body geometry of K. Therefore, the dynamic anisotropic risk field E generated by K... K Defined as:

[0116]

[0117] Among them, E K The direction is from the center of K to point j. and Let K be the acceleration in the forward direction and the acceleration in the normal direction, respectively. and Let μ be the acceleration coefficient, and μ0 be the peak field strength occurring at the center of K. and They are respectively The angle between M and the direction of travel of K and the normal, K To account for the equivalent mass of the vehicle type and speed:

[0118]

[0119] Where, m K For the true mass of K, v K Let b be the velocity in the forward direction of K. m and k m A constant related to the type of K.

[0120] By transforming between the coordinate system fixed on the vehicle and the world coordinate system, the established dynamic anisotropic risk field exhibits asymmetric field strength distributions in both the vehicle's direction of travel and its normal direction, and considers the influence of acceleration magnitude and direction on the risk posed by the vehicle. Furthermore, when the vehicle K does not intend to change lanes, the field strength E generated by K in the adjacent lane is...K It needs to be multiplied by the lane line filtering factor f L .

[0121] (2.2) Lane Line Filtering Effect: Lane lines constrain vehicle movement and have a strong filtering effect on the dynamic anisotropic risk field generated by traffic. This filtering effect is related to the lane line type and the vehicle's motion intention. When the vehicle's intention is to maintain its original lane, a lane line filtering factor f is defined. L :

[0122]

[0123] Among them, T L d is the lane line type coefficient. L,K Let d be the distance d is the distance between the edge of the vehicle K and the lane line. L,center This represents the distance between the edge of the vehicle and the lane line when K is traveling along the current lane centerline. However, when K intends to change lanes, the lane boundary between the original lane and the target lane will lose its filtering effect on the dynamic anisotropic risk field generated by K.

[0124] Appendix Figure 3 The dynamic anisotropic risk field generated by vehicles in different motion states is presented. (See attached diagram) Figure 3 As shown in (a)-(c), the risk field of a vehicle traveling in a straight line is symmetrically distributed in the normal direction and asymmetrically distributed in the forward direction, depending on its acceleration. A decelerating vehicle generates a stronger field behind it than in front of it, while an accelerating vehicle generates a stronger field in front of it than behind it. Furthermore, due to the filtering effect of lane markings, the field strength generated by a vehicle in an adjacent lane decays more rapidly than in its own lane; as shown in the attached diagram. Figure 3 As shown in (d)-(f), the electric field intensity generated by a turning vehicle exhibits an asymmetric distribution in both the forward direction and the normal direction. Due to centripetal acceleration, the electric field intensity generated by the vehicle on the inside of its turning trajectory is greater than that on the outside, and the electric field intensity generated in adjacent lanes is also affected by the lane line filtering effect; as shown in the appendix. Figure 3 As shown in (g)-(i), the electric field intensity generated by the lane-changing vehicle exhibits an asymmetric distribution in both the forward direction and the normal direction. Due to lateral acceleration, the electric field intensity generated by the vehicle inside its lane-changing trajectory is greater than that on the outside. Unlike turning vehicles, the lane line filtering effect of the boundary between the original lane and the target lane disappears because the vehicle has already expressed its intention to change lanes.

[0125] (2.3) Static Anisotropic Risk Field: The static anisotropic risk field is used to describe the risks posed by static obstacles in a traffic environment, such as roadblocks, cones, or stationary vehicles. A coordinate system X is established fixed on the static element S. S O S Y S O S Let S be the centroid, and OX be the centroid.S OY represents the direction of the major axis of the circumscribed ellipse S. S This represents the direction of the minor axis of the circumscribed ellipse S. The distance vector d in the world coordinate system... S,j Convert to X S O S Y S Equivalent distance vector in coordinate system

[0126]

[0127] in, and OX in world coordinate system S and OY S The unit vector, α S and β S OX S and OY S The distance scaling factor in the direction is related to the geometric dimensions of S. Referring to equation (3), the static anisotropic risk field E generated by S... S Defined as:

[0128]

[0129] Where, m S For the true mass of S, μ S The peak value of the static anisotropic risk field that appears at the S-center of mass is denoted as .

[0130] See appendix Figure 4 The paper presents the static anisotropic risk field generated by static obstacles of different types, shapes, and placement angles. It can be seen that the established static anisotropic risk field exhibits an asymmetric distribution according to the shape of the circumscribed ellipse of the static obstacle, with anisotropic field strength distribution in different directions. Furthermore, lane lines also have a filtering effect on the static anisotropic risk field; when a static obstacle does not cross a lane line, the field strength of the static anisotropic risk field it generates in other lanes is 0.

[0131] In a certain traffic environment, the total field strength E of the objective environment anisotropic risk field obtained by the objective environment anisotropic risk field calculation module is expressed as:

[0132]

[0133] Where M and N represent the number of vehicles and static obstacles in the traffic environment that affect the vehicle's driving safety, respectively. The objective environment anisotropic risk field calculation module selects a longitudinal range of 150m in front of and behind the vehicle for traffic participants, and a lateral range of the vehicle's current lane and the adjacent lanes to the left and right. In actual calculation, the objective environment anisotropic risk field calculation module performs rasterization processing on the selected range, calculates the field strength generated by traffic participants at each grid point within the selected range, and thus obtains the objective environment anisotropic risk field distribution.

[0134] (3) In order to match the risk area determined by the driver in the driver's field of vision, the objective environment anisotropic risk field field of vision transformation module projects the anisotropic risk field established by the objective environment anisotropic risk field calculation module in the world coordinate system to the driver's field of vision through coordinate transformation of world coordinate system-camera coordinate system-image coordinate system-pixel coordinate system, and outputs the anisotropic risk field in the driver's field of vision.

[0135] Appendix Figure 2 (b) The driver's forward field of vision is divided into six areas, while the left and right fields of vision specifically refer to the left and right rearward areas as observed through the left and right side mirrors, respectively. Therefore, see Appendix Figure 5 The dividing line is drawn according to the position of the front seat of the vehicle. The risk field in the left and right rear areas of the dividing line does not need to be transformed into a coordinate system, while the risk field in front of the dividing line needs to be projected onto the driver's forward field of vision in order to form a matching relationship with the six areas in the driver's forward field of vision.

[0136] The image acquisition angle of the front-facing camera in an intelligent vehicle can be approximately equivalent to the driver's forward field of vision. Therefore, the projection process of the anisotropic risk field is equivalent to the transformation from the world coordinate system to the front-facing camera pixel coordinate system. The objective environment anisotropic risk field field of vision transformation module defines four coordinate systems: the world coordinate system, the camera coordinate system, the image coordinate system, and the pixel coordinate system. Specifically:

[0137] The world coordinate system is a three-dimensional coordinate system that describes the position of an object in the real physical world, with the X-axis as the coordinate axis. w Y w and Z w The coordinate axes are perpendicular to each other, with units of meters (m). The camera coordinate system is a three-dimensional coordinate system established on the camera, with the camera's optical center O. c With the origin as the coordinate axis, X c Y c and Z c perpendicular to each other, X c and Y c X parallel to the image formed by the camera i axis and Y i Axial direction, Z cParallel to the camera's principal optical axis, with coordinate axes in meters (m); the image coordinate system is a two-dimensional coordinate system describing the image captured by the camera, with its origin O. i The intersection of the principal optical axis and the image plane, coordinate axis X i and Y i The axes are perpendicular to each other, and the unit of measurement is mm. The pixel coordinate system is a two-dimensional coordinate system that describes the position of a pixel on the image captured by the camera, with the origin at the image vertex O. UV The coordinate axes U and V are parallel to the X-axis of the image coordinate system. i axis and Y i Axis, the unit of coordinate axes is pixels.

[0138] Let the coordinates of a point P in the world coordinate system be (x... w ,y w ,z w If the coordinates are such that the transformation to the pixel coordinate system requires the following process:

[0139] (3.1) Transformation from World Coordinate System to Camera Coordinate System: This transformation is essentially a translation and rotation of a rigid body. First, the origin of the world coordinate system is translated to the origin of the camera coordinate system through a translation transformation. Then, the camera coordinate system is rotated around the three axes of the world coordinate system. The transformation from the world coordinate system to the camera coordinate system is as follows:

[0140]

[0141] Among them, (x c ,y c ,z c (t1, t2, t3) are the coordinates of point P in the camera coordinate system, (t1, t2, t3) are the positions of the origin of the camera coordinate system in the world coordinate system, and R is the rotation matrix. Further transformation of equation (9) yields the homogeneous matrix form:

[0142]

[0143] Where T is the translation matrix, C Int This is the extrinsic parameter matrix of the camera.

[0144] (3.2) Transformation from camera coordinate system to image coordinate system: This transformation requires projecting the three-dimensional image onto the two-dimensional image using perspective photography relationships. The transformation relationship is as follows:

[0145]

[0146] Among them, (x i ,y i ,1) represents the normalized coordinates of point P in the image coordinate system, and f is the camera focal length.

[0147] (3.3) Conversion from image coordinate system to pixel coordinate system: Both the pixel coordinate system and the image coordinate system are two-dimensional coordinate systems. The difference lies in the position of the origin and the units of measurement of the coordinate axes. The conversion relationship is as follows:

[0148]

[0149] Where (u,v,1) are the normalized coordinates of point P in the pixel coordinate system.

[0150] Combining equations (9) to (12), we can obtain the transformation relationship from the world coordinate system to the pixel coordinate system as follows:

[0151]

[0152]

[0153] Among them, C Ext This is the camera extrinsic parameter matrix.

[0154] As can be seen from equation (13), coordinate system transformation requires the camera intrinsic parameter matrix C. Int and camera extrinsic matrix C Ext Given that the camera's installation location and basic parameters are known, those skilled in the art can easily obtain C. Int and C Ext Therefore, through the transformation from the world coordinate system to the pixel coordinate system shown in equation (13), the anisotropic risk field of the objective environment can be transformed into the driver's field of vision. To establish a one-to-one correspondence with the driver's eight visual field regions, see Appendix. Figure 5 The anisotropic risk field of the objective environment under the driver's line of sight is divided into areas related to the surrounding environment. Figure 2 (b) shows the same 8 areas, with area 7 and area 8 representing the driver's left field of vision (left rearview mirror) and right field of vision (right rearview mirror), respectively.

[0155] (4) The intelligent vehicle risk area determination module based on driver intention takes the anisotropic risk field under the driver's field of vision and the driver's driving intention as input, and outputs the risk area determined by the intelligent vehicle according to the magnitude of the risk field and the driver's driving intention.

[0156] In this invention, the driver's driving intention is categorized into three types: left lane change, lane keeping, and right lane change. Currently, driver intention recognition is a mature technology, and those skilled in the art can directly utilize existing technology to construct the driver intention recognition module when building the system described in this invention. Therefore, this invention directly uses the driver's driving intention as a known input.

[0157] See appendix Figure 1The intelligent vehicle risk area determination module based on driver intent takes the anisotropic risk field under the driver's field of vision and the driver's driving intent as inputs, and outputs the risk area determined by the intelligent vehicle at this time. Specifically:

[0158] (4.1) First, the intelligent vehicle risk area determination module based on driver intention selects the areas that affect the driver's driving intention from 8 areas according to the driver's driving intention: if the driver's driving intention is to keep in the lane, then the areas that affect the driver's lane keeping are areas 1-6; if the driver's driving intention is to change lanes to the left, then the areas that affect the driver's left lane change are areas 1, 2, 4, 5, and 7; if the driver's driving intention is to change lanes to the right, then the areas that affect the driver's right lane change are areas 2, 3, 5, 6, and 8.

[0159] (4.2) Further, in areas that affect the driver's driving intention, if there are grid points in a certain area where the total field strength E of the objective environmental anisotropic risk field is greater than 0.5, then the intelligent vehicle risk area determination module based on the driver's driving intention will take that area as the risk area determined by the intelligent vehicle.

[0160] Incorporating the driver's driving intention into the risk area determination of intelligent vehicles can ensure that risks unrelated to the driver's driving intention will not interfere with the operation of the warning system, thereby effectively reducing the false warning rate and improving the acceptance, trust and adaptability of intelligent vehicle users to the system described in this invention.

[0161] (5) See Appendix Figure 1 The human-vehicle risk perception result characterization module based on the cumulative effect and decay effect of risk perception takes the risk area determined by the driver and the risk area determined by the intelligent vehicle as input, and quantifies the risk perception results of the driver and the intelligent vehicle respectively.

[0162] This invention posits that drivers perceive not only the spatial distribution of risk at the current moment but also the cumulative effect of risk over time. Previous research indicates that the average reaction time for a driver from perceiving their environment to taking driving action is 3.45 seconds; therefore, it can be assumed that the driver's actions are related to the risk perception results over at least the past 3.45 seconds. Consequently, the human-vehicle risk perception result characterization module based on the cumulative and decay effects of risk perception considers the cumulative effect of risk over the past 4 seconds when characterizing the risk perception results of both the driver and the intelligent vehicle.

[0163] At the current time t0, the driver's risk perception is related to all areas the driver has observed from t0-4 to t0. However, after observing a certain area, the driver's perceived risk level for that area gradually declines over time; this is what the present invention calls the risk perception decay effect.

[0164] Define dr k (t) is used to mark whether the region k at sampling time t is the effective region of the driver's risk perception (dr) k (t) = 1 means that dr k (t) = 0 represents no). The human-vehicle risk perception result characterization module based on the cumulative and decay effects of risk perception will dr k Treating the input signal as a step, the cumulative effect of driver risk perception is described by convolution operations based on a sliding time window, using a semi-Hanning window function H. h To describe the decay effect of driver risk perception, we use dr k and H h The convolution operation can be used to obtain the driver risk perception result (DR) that takes into account both cumulative and decay effects. k :

[0165]

[0166] Similarly, the human-vehicle risk perception result characterization module based on the cumulative and decay effects of risk perception applies the cumulative and decay effects of risk perception to the risk perception result IR of intelligent vehicles. k Construction:

[0167]

[0168] Among them, ir k (t) is used to mark whether region k at sampling time t is a risk region determined by the intelligent vehicle (ir) k (t) = 1 means that ir k (t) = 0 means it is not).

[0169] (6) The driver risk perception reliability quantification module takes the risk area determined by the driver, the risk area determined by the intelligent vehicle, and the risk perception results of the driver and the intelligent vehicle as input, and generates and outputs a driver risk perception reliability factor that can quantify the reliability of the driver's risk perception through region matching.

[0170] See appendix Figure 1 The driver risk perception reliability quantification module comprehensively calculates and outputs the Driver Risk Perception Reliability Factor (RPRF) based on the risk areas determined by the driver, the risk areas determined by the intelligent vehicle, the driver's risk perception results, and the intelligent vehicle's risk perception results, thereby quantifying the reliability of the driver's risk perception. The specific calculation process of RPRF is as follows:

[0171] (6.1) Matching of risk areas determined by the driver and the intelligent vehicle: This invention posits that during driving, the driver's gaze is mostly focused on the area directly in front of their field of vision, namely areas 2 and 5. When these two areas are deemed risk areas by the driver, the driver can also perceive other areas in their forward field of vision using peripheral vision, but the allocation of attention to different areas differs. When the driver's gaze is located in area 7 or 8, it indicates that the driver is observing the left or right rearview mirror outside the vehicle. At this time, it is difficult for the driver to perceive other areas besides area 7 or 8.

[0172] Based on this premise, the driver risk perception reliability quantification module divides the degree of matching r between the risk area determined by the driver and the risk area determined by the intelligent vehicle into 5 levels: high (r=1), medium-high (r=0.8), medium (r=0.5), medium-low (r=0.3), and low (r=0).

[0173] Appendix Figure 6 This paper presents the calculation rules for the risk area (r) determined by the driver, and the different risk areas determined by the intelligent vehicle. (See appendix.) Figure 6 In the diagram, circles represent the driver's effective risk perception area, and rectangles represent risk areas marked by AVS. When the intelligent vehicle simultaneously identifies multiple areas as risk areas, r takes the maximum value among them. When the driver's gaze is not captured in any of the eight areas, r is 0 for all areas.

[0174] (6.2) Driver Risk Perception Reliability Factor (RPRF) Calculation: The Driver Risk Perception Reliability Quantification Module defines the Driver Risk Perception Reliability Factor (RPRF) to quantitatively describe the driver's perceived risk reliability. The calculation of RPRF includes three cases:

[0175] ①If This indicates that the driver's gaze point was located at the adjacent point within the past 4 seconds. Figure 2 (b) If the driver’s attention is diverted or the gaze point cannot be detected outside the 8 areas shown in the diagram, it can be considered that the driver’s attention is diverted or that the driver cannot effectively perceive the environmental risks. At this time, the driver’s risk perception reliability is the lowest. The driver risk perception reliability quantification module sets RPRF=0.

[0176] ②If This indicates that the intelligent vehicle perceives no risk in the environment. This indicates that the driver has perceived a risk that the intelligent vehicle has not perceived. At this time, the driver's risk perception reliability is the highest, and the driver risk perception reliability quantification module sets RPRF=1.

[0177] ③If and This indicates that both the driver and the intelligent vehicle have effectively perceived the risks in the environment. Therefore, the driver risk perception reliability quantification module needs to further calculate the driver risk perception reliability based on the risk perception results of the driver and the intelligent vehicle. The driver risk perception reliability quantification module sets two parameters, δ1 and δ2, for calculating the driver risk perception reliability factor (RPRF).

[0178] The δ1 is used to characterize the three main driver visual directions: forward, left, and right (as shown in the appendix). Figure 2 Driver risk perception as shown in (a):

[0179]

[0180] in, and These represent the matching degree of human-vehicle risk perception in the driver's forward, left, and right visual fields, respectively. and The calculation rules are shown in Appendix Table 1-3.

[0181] Based on using δ1 to characterize the driver's risk perception in the three main directions of the driver's field of vision, the driver risk perception reliability quantification module further sets δ2 to characterize the secondary directions. Figure 2 (b) Driver risk perception in the eight specific visual areas shown:

[0182]

[0183] Where N is the number of sampling points contained in [t0-4, t0]. As can be seen from equation (18), the range of δ2 is [0, 1].

[0184] Based on δ1 and δ2, the driver risk perception reliability quantification module calculates the driver risk perception reliability factor RPRF using the following formula:

[0185] RPRF=δ1·δ2 (19)

[0186] Based on the design process of δ1 and δ2 above, we know that the value range of δ1 is 0 or 1, and the value range of δ2 is [0, 1]. Therefore, it is easy to know that the value range of RPRF is [0, 1].

[0187] (7) The graded early warning module based on driver risk perception reliability sets three levels of thresholds and divides the early warning level into four levels according to the driver risk perception reliability factor, thus completing the intelligent vehicle graded early warning starting from the driver risk perception layer.

[0188] See appendix Figure 7The graded early warning module based on driver risk perception reliability uses the driver's risk perception reliability factor (RPRF) as its basis and issues warnings at four levels. The module has three warning thresholds: RPRF1, RPRF2, and RPRF3, with the following values:

[0189]

[0190] The four levels of warning methods and activation ranges in the graded warning module based on the reliability of driver risk perception are as follows:

[0191] (7.1) When RPRF > RPRF1, it indicates that the driver's risk perception reliability is extremely high, and the driver can even perceive risks that the intelligent vehicle does not perceive. Therefore, the driver's driving behavior is considered reliable, and the graded warning module based on the reliability of the driver's risk perception does not need to issue a warning.

[0192] (7.2) When RPRF2 < RPRF ≤ RPRF1, it indicates that the driver's risk perception reliability has decreased. It can be considered that the driver has not fully perceived the environmental risks and is prone to slightly risky driving behaviors. At this time, the graded warning module based on the driver's risk perception reliability will guide the driver to pay attention to one or more previously unnoticed risk areas through visual cues, so as to correct the driving behavior in a timely manner. When the graded warning module based on the driver's risk perception reliability needs to prompt the driver to pay attention to a certain area of ​​the forward field of vision, it can project a prompt signal to the area of ​​the forward field of vision that needs to be paid attention to through the currently mature head-up display system; when the graded warning module based on the driver's risk perception reliability needs to prompt the driver to pay attention to the left or right field of vision, it can directly prompt the driver through the flashing of the warning lights on the left or right rearview mirrors outside the vehicle.

[0193] (7.3) When RPRF3 < RPRF ≤ RPRF2, it indicates that the driver's risk perception reliability has decreased to a certain level, and the driver is highly prone to dangerous driving behavior. At this time, the graded warning module based on the driver's risk perception reliability will warn the driver through more intense sensory cues. When the driver's driving intention is to maintain lane position, the graded warning module based on the driver's risk perception reliability warns the driver through slight braking; when the driver's driving intention is to change lanes, the graded warning module based on the driver's risk perception reliability warns the driver by applying steering wheel damping (i.e., torque opposite to the driver's current steering behavior).

[0194] (7.4) When RPRF≤RPRF3, it indicates that the driver's risk perception reliability is extremely low, and the driver's driving behavior is carried out without effectively perceiving environmental risks. It can be considered that the driver's driving behavior is extremely dangerous. At this time, the graded warning module based on the reliability of the driver's risk perception will perform emergency avoidance. When the driver's driving intention is to keep in the lane, the graded warning module based on the reliability of the driver's risk perception will perform emergency braking to reduce the vehicle speed to a safe range or even stop it directly. When the driver's driving intention is to change lanes, the graded warning module based on the reliability of the driver's risk perception will forcibly interrupt the driver's lane-changing behavior and control the vehicle to return to the original lane.

[0195] Appendix 1 Calculation rules

[0196]

[0197] Appendix 2 Calculation rules

[0198]

[0199] Appendix 3 Calculation rules

[0200]

Claims

1. An intelligent vehicle graded early warning system based on the reliability of driver risk perception, characterized in that: It includes a driver risk area determination module based on eye-tracking information, an objective environment anisotropic risk field calculation module, an objective environment anisotropic risk field field view conversion module, an intelligent vehicle risk area determination module based on driver intent, a human-vehicle risk perception result representation module based on risk perception cumulative effect and decay effect, a driver risk perception reliability quantification module, and a graded early warning module based on driver risk perception reliability. (1) The driver risk area determination module based on eye movement information takes the driver's eye movement information as input and outputs the risk area determined by the driver according to the location of the driver's visual attention point; (2) The objective environment anisotropic risk field calculation module takes traffic environment information as input and outputs anisotropic risk field in the world coordinate system; the objective environment anisotropic risk field calculation module receives traffic environment information and describes the objective risks generated by traffic participants by establishing anisotropic risk field based on the basic types, geometric shapes, defined attributes and motion states of traffic participants; the anisotropic risk field is divided into dynamic anisotropic risk field for describing moving traffic vehicles, static anisotropic risk field for describing static obstacles and lane line filtering effect; (3) The objective environment anisotropic risk field field of view conversion module transforms the anisotropic risk field in the world coordinate system to the driver's field of view through coordinate transformation of the world coordinate system-camera coordinate system-image coordinate system-pixel coordinate system, and outputs the anisotropic risk field in the driver's field of view. (4) The intelligent vehicle risk area determination module based on driver intention takes the anisotropic risk field under the driver's field of vision and the driver's driving intention as input, and outputs the risk area determined by the intelligent vehicle according to the magnitude of the risk field and the driver's driving intention. The driver's driving intentions include changing lanes to the left, maintaining lane position, and changing lanes to the right; (5) The human-vehicle risk perception result representation module based on the cumulative effect and decay effect of risk perception takes the risk area determined by the driver and the risk area determined by the intelligent vehicle as input, and represents the cumulative effect of risk perception by statistically analyzing the human-vehicle risk perception area over a period of time, and represents the decay effect of risk perception by spatiotemporal convolution operation based on the semi-Hanning window function, thereby representing and outputting the risk perception results of the driver and the intelligent vehicle. The human-vehicle risk perception result characterization module based on the cumulative and decay effects of risk perception considers the cumulative and decay effects of risk over a past period when describing the risk perception results of drivers and intelligent vehicles; [Definition] To mark the sampling time area Is it within the driver's effective risk perception zone? The representative is, The representative is not; the human-vehicle risk perception result characterization module based on the cumulative and decay effects of risk perception will... Treating the input signal as a step, the cumulative effect of driver risk perception is described by convolution operations based on a sliding time window, using a semi-Hanning window function. To describe the decay effect of driver risk perception, we use... and The convolution operation yields driver risk perception results that take into account both cumulative and decay effects. : in, Represents the current moment; The human-vehicle risk perception result characterization module based on the cumulative and decay effects of risk perception applies the cumulative and decay effects of risk perception to the risk perception results of intelligent vehicles. Construction: in, Used to mark sampling time area Is it a risk area for intelligent vehicles? The representative is, This means no; (6) The driver risk perception reliability quantification module takes the risk area determined by the driver, the risk area determined by the intelligent vehicle, and the risk perception results of the driver and the intelligent vehicle as input, and generates and outputs a driver risk perception reliability factor that can quantify the reliability of the driver risk perception through region matching. (7) The graded early warning module based on driver risk perception reliability sets three levels of thresholds and divides the early warning level into four levels according to the driver risk perception reliability factor, thus completing the intelligent vehicle graded early warning starting from the driver risk perception layer.

2. The intelligent vehicle graded early warning system based on driver risk perception reliability according to claim 1, characterized in that: The driver's field of vision is divided into three main directions: forward, left, and right. The forward field of vision is further divided into six regions: upper left, upper, upper right, lower left, lower, and lower right, which are numbered as regions 1-6. The left and right fields of vision are numbered as regions 7 and 8, respectively.

3. A graded early warning system for intelligent vehicles based on the reliability of driver risk perception, as described in claim 1 or 2, characterized in that: The driver risk area determination module based on eye-tracking information takes driver eye-tracking information as input. The driver's eye-tracking behavior includes fixation behavior and saccade behavior. The eye-tracking device collects the driver's eye-tracking information and captures the driver's fixation behavior from it, thereby obtaining the area where the fixation point is located. When the driver fixates on a certain area, that area is the risk area determined by the driver. The driver risk area determination module based on eye-tracking information determines from the driver's eye-tracking information collected by the eye-tracking device that the driver's fixation point falls on one or more areas of his / her field of vision, thereby obtaining the corresponding number of the risk area determined by the driver.

4. The intelligent vehicle graded early warning system based on driver risk perception reliability according to claim 1, characterized in that: (2.1) The anisotropy of the dynamic anisotropic risk field is reflected in the motion state of the vehicle. In a global coordinate system, transportation vehicles With a certain point in the field Distance vector between for: (1) Establish a fixed connection coordinate system on ,in for The center of mass, represent The direction of progress represent The normal direction of the forward movement; the world coordinate system Convert to Equivalent distance vector in coordinate system : , (2) in, and respectively in world coordinate system and unit vector, and They are respectively and The distance scaling factor in the direction, and It is related to the vehicle's geometric dimensions; then The resulting dynamic anisotropic risk field Defined as: (3) in, The direction is from Center pointing point , and They are respectively Acceleration in the forward direction and normal direction, and For acceleration coefficient, For appearing Peak field strength at the center and They are respectively and The angle between the direction of travel and the normal direction. To account for the equivalent mass of the vehicle type and speed: , (4) in, for The true quality for forward direction and speed and To and Constants related to the type; In addition, when the shuttle bus When there is no intention to change lanes, Field strength generated in adjacent lanes It needs to be multiplied by the lane line filter factor. ; The lane line filtering effect described in (2.2) is related to the lane line type and the vehicle's motion intention. When the vehicle's intention is to maintain its original lane, a lane line filtering factor is defined. : , (5) in, Lane type coefficient, For shuttle bus The distance between the edge of the vehicle body and the lane line. For shuttle bus The distance between the edge of the vehicle body and the lane line when traveling along the current lane centerline; however, when a vehicle intends to change lanes, the lane boundary between the original lane and the target lane will lose its filtering effect on the dynamic anisotropic risk field generated by the vehicle. (2.3) The static anisotropic risk field described above is used to describe the risks generated by static obstacles in the traffic environment, and is established on static elements. coordinate system on ,in for Center of mass represent The direction of the major axis of the circumscribed ellipse, represent The direction of the minor axis of the circumscribed ellipse; the distance vector in the world coordinate system. Convert to Equivalent distance vector in coordinate system : , (6) in, and respectively in world coordinate system and unit vector, and They are respectively and The distance scaling factor in the direction, and It is related to the geometric dimensions; refer to equation (3), The resulting static anisotropic risk field Defined as: , (7) in, for The true quality For appearing Peak value of the static anisotropic risk field strength at the center of mass; In a specific traffic environment, the total field strength of the anisotropic risk field of the objective environment obtained by the objective environment anisotropic risk field calculation module is... Represented as: , (8) Wherein, M and N are the number of vehicles and static obstacles in the traffic environment that affect the driving safety of the vehicle, respectively; the objective environment anisotropic risk field calculation module selects the longitudinal range of traffic participants as 150m in front of and behind the vehicle, and the lateral range as the vehicle's current lane and the adjacent lanes to the left and right of the current lane.

5. A graded early warning system for intelligent vehicles based on the reliability of driver risk perception, as described in claim 1 or 4, characterized in that: The objective environment anisotropic risk field field of view conversion module transforms the anisotropic risk field in the world coordinate system to the driver's field of view through coordinate transformation from the world coordinate system to the camera coordinate system to the image coordinate system to the pixel coordinate system, and outputs the anisotropic risk field in the driver's field of view. Among them, the dividing line is drawn according to the position of the front seat of the vehicle. The risk field in the left and right rear areas of the dividing line does not need to be transformed into a coordinate system. The risk field in front of the dividing line is projected onto the driver's forward vision and forms a matching relationship with the 6 areas in the driver's forward vision. The objective environment anisotropic risk field view transformation module defines four coordinate systems: world coordinate system, camera coordinate system, image coordinate system, and pixel coordinate system, wherein: The world coordinate system is a three-dimensional coordinate system that describes the position of an object in the real physical world, with coordinate axes... , and The coordinate axes are perpendicular to each other, with units of meters (m). The camera coordinate system is a three-dimensional coordinate system established on the camera, centered at the camera's optical center. Origin, coordinate axes , and perpendicular to each other, and Parallel to the image formed by the camera shaft and Axial direction, Parallel to the camera's principal optical axis, with coordinate axes in meters (m); the image coordinate system is a two-dimensional coordinate system describing the image captured by the camera, with the origin at... The intersection of the principal optical axis and the image plane, coordinate axes and The axes are perpendicular to each other, and the unit of measurement is mm. The pixel coordinate system is a two-dimensional coordinate system that describes the position of a pixel on the image captured by the camera, with the origin at the image vertex. coordinate axes and Parallel to the image coordinate system shaft and Axis, the unit of coordinate axes is pixel; Let a point in the world coordinate system The coordinates are The transformation to the pixel coordinate system requires the following process: (3.1) Transformation from world coordinate system to camera coordinate system: First, the origin of the world coordinate system is translated to the origin of the camera coordinate system through a translation transformation. Then, the camera coordinate system is rotated around the three axes of the world coordinate system. The transformation from world coordinate system to camera coordinate system is as follows: , (9) in, It is a point Coordinates in the camera coordinate system It is the position of the camera coordinate system origin in the world coordinate system. The rotation matrix is ​​used; transforming equation (9) yields the homogeneous matrix form: , (10) in, It is a translation matrix. The extrinsic parameter matrix of the camera; (3.2) Transformation from camera coordinate system to image coordinate system: The 3D coordinates are projected to 2D using perspective photography relationships. The transformation relationship is as follows: , (11) in, For point Normalized coordinates in the image coordinate system The focal length of the camera; (3.3) Transformation from image coordinate system to pixel coordinate system: The transformation relationship is as follows: , (12) in, For point Normalized coordinates in pixel coordinate system; Combining equations (9) to (12), the transformation relationship from the world coordinate system to the pixel coordinate system is obtained as follows: , (13) , (14) in, This is the camera extrinsic parameter matrix; Through the transformation from the world coordinate system to the pixel coordinate system shown in equation (13), the anisotropic risk field of the objective environment is transformed into the driver's field of vision; The anisotropic risk field of the objective environment under the driver's field of vision is divided into 8 regions that are the same as the driver's field of vision. Region 7 and Region 8 represent the driver's left field of vision and right field of vision, respectively.

6. The intelligent vehicle graded early warning system based on driver risk perception reliability according to claim 1, characterized in that: The intelligent vehicle risk area determination module based on driver intent takes the anisotropic risk field under the driver's field of vision and the driver's driving intent as input: (4.1) First, the intelligent vehicle risk area determination module based on driver intention selects the area that affects the driver's driving intention from 8 visual areas according to the driver's driving intention; (4.2) In areas that affect the driver's driving intentions, if there is an anisotropic risk field with a total field strength in a certain area, the total field strength of the risk field is considered. If the number of grid points is greater than 0.5, then the intelligent vehicle risk area determination module based on the driver's driving intention will identify that area as a risk area for the intelligent vehicle.

7. The intelligent vehicle graded early warning system based on driver risk perception reliability according to claim 1, characterized in that: The driver risk perception reliability quantification module comprehensively calculates and outputs the driver risk perception reliability factor based on the risk areas determined by the driver, the risk areas determined by the intelligent vehicle, the driver's risk perception results, and the intelligent vehicle's risk perception results. This allows for a quantitative description of the reliability of drivers' risk perception. The specific calculation process is as follows: (6.1) Matching of risk areas determined by the driver and the intelligent vehicle: The driver risk perception reliability quantification module divides the matching degree r between the risk areas determined by the driver and the risk areas determined by the intelligent vehicle into 5 levels: high, medium-high, medium, medium-low, and low; when the intelligent vehicle determines multiple areas as risk areas at the same time, r takes the maximum value among them; when the driver's gaze point is not captured in any of the 8 visual fields, r for all areas is 0. (6.2) Driver risk perception reliability factor Calculation: The driver risk perception reliability quantification module defines the driver risk perception reliability factor. To quantitatively describe the reliability of driver risk perception, The calculation involves three cases: ①If If the driver's gaze point is outside the eight areas of their field of vision or cannot be detected, it indicates that the driver's attention is distracted or they are unable to effectively perceive environmental risks. In this case, the driver's risk perception reliability is at its lowest. The driver risk perception reliability quantification module then... ; ②If This indicates that the intelligent vehicle perceives no risk in the environment. This indicates that the driver perceived a risk that the intelligent vehicle did not. At this point, the driver's risk perception reliability is at its highest. The driver risk perception reliability quantification module then... ; ③If and If the result is positive, it means that both the driver and the intelligent vehicle have effectively perceived the risks in the environment. Therefore, the driver risk perception reliability quantification module needs to calculate the driver risk perception reliability based on the risk perception results of the driver and the intelligent vehicle. The driver risk perception reliability quantification module is set and Two parameters are used for the driver's risk perception reliability factor. Calculation; The Used to characterize driver risk perception in the three main driver visual directions: forward, left, and right. , (17) in, , and These represent the matching degree of human-vehicle risk perception in the driver's forward, left, and right visual fields, respectively. The value range is 0 or 1; In use Based on characterizing the driver's risk perception in the three main directions of the driver's field of vision, the driver risk perception reliability quantification module is set up... To characterize the driver's risk perception in eight specific areas of the driver's field of vision: ,(18) in, for The number of sampling points contained within; The range of values ​​is ; according to and The driver risk perception reliability quantification module calculates the driver risk perception reliability factor using the following formula. : (19) The range of values ​​is .

8. The intelligent vehicle graded early warning system based on driver risk perception reliability according to claim 1, characterized in that: The graded early warning module based on the reliability of driver risk perception uses the driver's risk perception reliability factor. Based on this, warnings are issued at four levels, and the graded warning module based on the reliability of driver risk perception has three warning thresholds: , and Its possible values ​​are as follows: (20) The four levels of warning methods and activation ranges in the graded warning module based on the reliability of driver risk perception are as follows: (7.1) When If the driver's risk perception reliability is extremely high, and they are even able to perceive risks that the intelligent vehicle does not perceive, then the driver's driving behavior is considered reliable. Therefore, the graded warning module based on the reliability of the driver's risk perception does not need to issue a warning. (7.2) When If the driver's risk perception reliability is reduced, it means that the driver has not fully perceived the environmental risks and is prone to slightly risky driving behaviors. At this time, the graded warning module based on the reliability of the driver's risk perception will guide the driver to pay attention to one or several risk areas that were not previously noticed through visual prompts, so as to correct the driving behavior in time. (7.3) When When the driver's risk perception reliability has decreased to a certain level, the driver is very likely to engage in dangerous driving behavior. At this time, the graded warning module based on the driver's risk perception reliability will warn the driver through more intense tactile cues. (7.4) When This indicates that the driver's risk perception reliability is extremely low at this time, and the driver's driving behavior is carried out without effectively perceiving environmental risks. The driver's driving behavior is considered extremely dangerous, and the graded early warning module based on the reliability of the driver's risk perception will execute emergency avoidance.

Images