A method and system for predicting illegal road occupation on curved roads

Abstract

The invention discloses a method for predicting illegal road occupation on a curve, and relates to the technical field of road safety prediction. The method comprises: constructing a visual road linear layer to extract the road linear shape perceived by a driver; constructing a driver dynamic visual layer to lay a foundation for quantifying the construction of the driver's visual characteristics; constructing a visual semantic layer to segment the visual road environment and extract semantic parameters in the visual road environment; constructing a visual depth layer to extract depth information of the visual road environment; establishing a visual sensitivity layer to extract a sensitive distance; obtaining whether a vehicle is speeding, and in addition obtaining parameters such as the driver's age, driving experience and gender; processing the parameters extracted before the driver passes the curve by an XGBoost machine learning algorithm, so as to predict whether the driver will illegally occupy the road when passing the curve of a rural two-lane road; the invention uses a highly interpretable prediction model to provide a reference for optimizing the road environment design and reduce the occurrence of illegal road occupation on the curve section of rural roads.

Description

Technical Field

[0001] The present invention relates to the technical field of road safety prediction, and in particular to a method and system for predicting illegal road occupation on a curve. Background Art

[0002] Due to the large curvature and narrow width of rural roads, many drivers illegally cross the road. When oncoming vehicles suddenly appear in the opposite lane, due to the short sight distance on the curved road section, drivers who illegally cross the road face the challenge of responding in time, which increases the possibility of head-on collision. This behavior is extremely risky and causes up to 60% of fatal cross-centerline collisions.

[0003] There are several countermeasures that can reduce illegal lane-sweeping on curved sections of two-lane rural roads. For road infrastructure, installing median barriers is a generally proven cost-effective method to reduce cross-centerline collisions, but installing median barriers increases the frequency of collisions that hit the median barriers. In addition to the above infrastructure-related methods, vehicle-based departure warning systems can also help reduce illegal lane-sweeping behaviors. Lane departure warning (LDW) is a safety system that uses a camera to identify lane markings to help drivers stay in their lanes. However, LDW is difficult to be effective when lane markings on rural roads are missing or blurred. In addition, LDW is a post-event warning system based on the driver's current driving position. Therefore, in order to give drivers enough time to react to the warning, an early warning system that can predict illegal lane-sweeping on curved sections of two-lane rural roads is necessary.

[0004] In addition, given the common phenomenon of lack of lane markings on rural roads, a system that can provide warnings without relying on cameras to identify lane markings is also needed. Predicting illegal lane occupation on curves on two-lane rural roads is a key step in the warning system. There are some studies on lane departure prediction models, most of which predict the upcoming lateral position of the vehicle by vehicle speed and vehicle lateral position. However, previous prediction models did not take into account the driver's visual perception. In fact, the visual information perceived by the driver is also important, accounting for more than 90% of the total information while driving. But the road environment information perceived by the driver's vision is not the same as the actual road environment information, and the driver's behavior mainly depends on the information perceived by the driver's vision, not the actual information. Summary of the invention

[0005] In view of the problem that the prediction model in the prior art does not take the driver's visual perception into consideration, the present invention provides a method and system for predicting illegal lane occupation on a curve. Starting from the driver's vision, it predicts whether the driver will illegally occupy the lane before the driver passes the curve, thereby solving the problems existing in the prior art.

[0006] A method for predicting illegal road occupation on a curved road comprises the following steps:

[0007] Based on the visual characteristics of the driver, a visual road alignment layer is constructed, and road alignment parameters that reflect the driver's perception are obtained through the visual road alignment layer;

[0008] Based on the change of the driver's dynamic field of view with speed, a driver's dynamic visual layer is constructed, and the dynamic change of the driver's field of view with speed is obtained through the driver's dynamic visual layer;

[0009] Based on semantic segmentation technology, a visual semantic layer is constructed, and the visual road environment is segmented through the visual semantic layer to obtain the components of the road environment visually perceived by the driver;

[0010] Based on the depth estimation technology, a visual depth layer is constructed to obtain the depth information of the road environment perceived by the driver through the visual depth layer;

[0011] Through the gradient weighted class activation mapping algorithm, a visual sensitivity layer is constructed, and the distance between the driver's visual sensitivity area and the field of vision is calculated through the visual sensitivity layer, so as to obtain the driver's visual sensitivity distance;

[0012] The road linear parameters, the dynamic changes of the driver's field of view with speed, the components of the road environment, the depth information of the road environment and the sensitive distance of the driver's vision are input into the prediction model of the curved section IROL of the two-lane highway constructed by the XGBoost machine learning algorithm, and the processing results are output. Based on the processing results, the road occupation situation of the driver when passing the curve of the rural two-lane road is predicted.

[0013] Furthermore, the road linear parameters reflecting the driver's perception include visual curve curvature and visual curve length.

[0014] Furthermore, the calculation formulas for the visual curve curvature and the visual curve length are:

[0015] v iL =S (i+1)L -S iL

[0016]

[0017] v iR =S (i+1)R -S iR

[0018]

[0019] Where i = 1, 2, 3; S iL At the control point P iL The cumulative length of the left visual road boundary at S (pixels); iR At the control point PiR The cumulative length of the right visual road boundary at (pixels); f iL is the left visual road boundary at control point P iL The tangent angle at (radians); f iR is the control point P iR The right visual road boundary at (radians); vS iL is the control point P iL and P (i+1)L (pixels) The length of the left visual curve between; vK iL is the control point P iL and P (i+1)L The curvature of the left visual curve between (radians / pixel); vS iR is the control point P iR and P (i+1)R (p (pixel) right visual curve length; vK iR is the control point P iR and P (i+1)R The curvature of the right visual curve between (radians / pixel).

[0020] Furthermore, the visual road environment components include: sky, road, protective facilities, traffic signs, forest and roadside cliffs; the depth values ​​of the road environment components are respectively represented in the visual depth layer through the depth value estimation technology.

[0021] Furthermore, the visual semantic layer and the visual depth layer both adopt feature extraction and feature fusion methods, extract features from an input image representing the driver's visual perception of the road environment through a residual network, and then fuse the extracted features through a feature pyramid network.

[0022] Furthermore, the distance calculation process between the driver's visual sensitivity area and the field of view is:

[0023]

[0024]

[0025] Among them, M i represents the i-th feature map, α i is its corresponding weight, ReLU is the rectified linear unit, c1 and c2 are the sizes of the feature maps, Z = c1 × c2, y is the model calculated based on the input image, Represents the feature map M i The pixel value at the jth row and kth column in .

[0026] Furthermore, the calculation process of the sensitive distance is:

[0027]

[0028]

[0029] Among them, Dist is the sensitive distance; α i is the sensitivity of the region calculated by the gradient weighted class activation mapping algorithm; β is the threshold of the weight, when α i When it is greater than β, the area is a visually sensitive area; ω is the weight of the visually sensitive area of ​​the entire visual road environment; d i is the distance between the ith visual sensitive area and the visual field.

[0030] Furthermore, the prediction model establishment process of the two-lane rural highway curved section IROL is as follows:

[0031] Given a model with m features (x i ∈R^m,y i ∈R), a dataset with n samples and K trees (x i ,y i ), calculate the predicted value of the XGBoost model:

[0032]

[0033] where h K (x i ) is the Kth decision tree, F K (x i ) is a strong learner of K-tree, F K-1 (x i ) is the strong learner of K-1 tree, 77 is the learning rate;

[0034] Minimize the objective function:

[0035]

[0036]

[0037] In the formula is the loss function, is the regularization term, y i is the true value, J is the number of leaves, w tj is the optimal value of the jth leaf node of the tth decision tree, λ is the parameter of the L2 regularization term, and γ is the parameter that controls the number of leaves;

[0038] The second-order Taylor expansion is used for the objective function:

[0039]

[0040] Among them G tj is the sum of the first-order derivatives of each leaf node; Htj is the sum of the second-order derivatives of each leaf node. When the derivative of the objective function is 0, w tj The optimal value is -G tj / H tj +λ.

[0041] Furthermore, a prediction system for illegal road occupation on a curved road comprises:

[0042] A visual road alignment module is used to construct a visual road alignment layer based on the driver's visual characteristics, and obtain road alignment parameters that reflect the driver's perception through the visual road alignment layer;

[0043] Driver dynamic vision module: used to construct a driver dynamic vision layer based on the change of the driver's dynamic vision with speed, and obtain the dynamic change of the driver's vision with speed through the driver dynamic vision layer;

[0044] Visual semantic module: used to build a visual semantic layer based on semantic segmentation technology, segment the visual road environment through the visual semantic layer, and thus obtain the components of the road environment visually perceived by the driver;

[0045] Visual depth module: used to construct a visual depth layer based on depth estimation technology, and obtain the depth information of the road environment perceived by the driver through the visual depth layer;

[0046] Visual Sensitivity Distance Module: It is used to construct a visual sensitivity layer through the gradient weighted class activation mapping algorithm, and calculate the distance between the driver's visual sensitivity area and the field of view through the visual sensitivity layer, so as to obtain the driver's visual sensitivity distance;

[0047] Prediction module: It is used to input road linear parameters, dynamic changes of driver's field of view with speed, components of road environment, depth information of road environment and sensitive distance of driver's vision into the prediction model of two-lane highway curved section IROL constructed by XGBoost machine learning algorithm, output the processing results, and predict the road occupation situation of the driver when passing the curve of rural two-lane road based on the processing results.

[0048] Furthermore, it also includes:

[0049] Explanation module: XGBoost is processed through the SHAP interpretable learning algorithm to further analyze whether the obtained parameters affect the illegal occupation of curves on rural two-lane roads.

[0050] The present invention provides a method and system for predicting illegal road occupation on a curved road, which has the following beneficial effects:

[0051] The present invention processes the road linear parameters, the dynamic changes of the driver's field of view with speed, the components of the road environment, the depth information and the distance between the sensitive area and the field of view according to the prediction model, fully considering the visual characteristics of the driver and the influence of the characteristics of the visual road environment on the driver, so that the extracted parameters are more in line with the driver's behavior; extracting parameters from the visual road environment can solve the problem of difficult prediction due to poor infrastructure construction such as blurred lane lines on low-grade highways; using machine learning methods to establish a high-accuracy prediction model, improve the applicability of the model, and use a highly interpretable prediction model to provide a reference for optimizing road environment design. BRIEF DESCRIPTION OF THE DRAWINGS

[0052] Figure 1 A flowchart of establishing a visual road linear layer in an embodiment of the present invention;

[0053] Figure 2 A flowchart of establishing a driver's dynamic vision layer in an embodiment of the present invention;

[0054] Figure 3 A flowchart of establishing a visual semantic layer and a visual depth layer in an embodiment of the present invention;

[0055] Figure 4 A flowchart of establishing a visual sensitivity layer in an embodiment of the present invention;

[0056] Figure 5 is a flowchart of interpretable analysis based on SHAP in an embodiment of the present invention;

[0057] Figure 6 This is a schematic diagram of a visual road linear layer in an embodiment of the present invention;

[0058] Figure 7 This is a schematic diagram of a dynamic visual layer in an embodiment of the present invention;

[0059] Figure 8 Schematic diagram of visual semantic layer and visual depth layer in an embodiment of the present invention;

[0060] Fig. 9 Schematic diagram of a visually sensitive layer in an embodiment of the present invention. DETAILED DESCRIPTION

[0061] The technical solutions in the embodiments of the present invention will be described clearly and completely below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, rather than all the embodiments.

[0062] The present invention discloses a method for predicting illegal road occupation on a rural two-lane road based on driver vision and interpretability; the steps are as follows:

[0063] Step 1: Based on the driver's visual characteristics, a visual road alignment layer is constructed to extract road alignment parameters that reflect the driver's perception; the road alignment parameters that reflect the driver's perception include visual curve curvature and visual curve length.

[0064] During driving, the driver can always perceive the "road alignment", even if the lane markings are unclear or blocked. These "road alignments" are regarded as the driver's visual road alignment, which comes from the driver's visual perception of the actual alignment; however, the visual road alignment is different from the actual alignment, and the driver mostly decides his behavior based on visual perception. Perspective mapping allows the visual road alignment to be converted into the actual alignment, which can be used to optimize the actual alignment through our analysis of the visual alignment. In order to better analyze the route from the driver's perception, a visual road alignment layer is proposed to quantify the characteristics of the visual road alignment. The present invention uses Catmull-Rom spline curves to construct the visual road alignment layer. Catmull-Rom spline curves pass through all control points and are both efficient and accurate in fitting the visual road alignment. In addition, the local controllability of the Catmull-Rom spline curve, that is, the local change of the control point will not affect other parts, also enhances its convenience in controlling the curve shape. Therefore, the present invention uses Catmull-Rom spline curves to fit the left and right visual road boundaries of the curved section of a two-lane rural highway.

[0065] like Figure 6 As shown in Figure 1, the visual road linear layer constructs a coordinate system with the lower left corner of the driver's field of view as the origin. The left spline curve has four control points (P 1L , P 2L , P 3L , P 4L ), the right spline curve also has four ((P 1R , P 2R , P 3R , P 4R ) control points to fit the shape of the road boundary. iL and S iR is the cumulative length (in pixels) of the left and right visual road boundaries at the above control points. In addition, f iL and f iR represents the tangent angle (in radians) of the left and right visual road boundaries at each control point. The horizontal connection of the spline control points on both sides divides the road into three feature regions: "near scene", "middle scene" and "far scene". These feature regions have different visual characteristics, which help to quantify the driver's perception of the visual road line shape. The curve length and curvature in the three feature regions of the left visual road boundary are extracted as shape parameters, called the left visual curve length and the left visual curve curvature, expressed as [vS iL , vK iL](i=1,2,3). Meanwhile, the shape parameters of the right visual road boundary are given by [vS iR , vK iR ] represents (i=1, 2, 3). Their calculation formulas are as follows:

[0066] v iL =S (i+1)L -S iL (1)

[0067]

[0068] v iR =S (i+1)R -S iR (3)

[0069]

[0070] Where i = 1, 2, 3; S iL At the control point P iL The cumulative length of the left visual road boundary at S (pixels); iR At the control point P iR The cumulative length of the right visual road boundary at (pixels); f iL is the left visual road boundary at control point P iL The tangent angle at (radians); f iR is the control point P iR The right visual road boundary at (radians); vS iL is the control point P iL and P (i+1)L (pixels) The length of the left visual curve between; vK iL is the control point P iL and P (i+1)L The curvature of the left visual curve between (radians / pixel); vS iR is the control point P iR and P (i+1)R (p (pixel) right visual curve length; vK iR is the control point P iR and P (i+1)R The curvature of the right visual curve between (radians / pixel).

[0071] Step 2: Based on the changes in the driver's dynamic field of view with speed, a driver's dynamic visual layer is constructed to lay the foundation for quantifying the driver's visual characteristics.

[0072] like Figure 7As shown in the figure, the dynamic visual layer indicates that the driver's field of view changes dynamically with speed. When the driver's speed is 40 km / h, the center of the field of view is located 180 meters away and the viewing angle is 100°. As the speed increases to 100 km / h, the position of the center of the field of view increases to 564 meters away, but the viewing angle decreases to 40°, which means that the driver's field of view shrinks as the speed increases, and their attention is more compressed to the farther center of the road. According to the different speeds of the driver, the road environment design needs to provide more effective information in the driver's corresponding field of view to facilitate the driver's perception.

[0073] Step 3: Based on semantic segmentation technology, a visual semantic layer is constructed to segment the visual road environment. The area ratios of the sky, roads, protective facilities and forests in the visual road environment, as well as the presence of traffic signs and roadside cliffs are extracted. The Python interface of OpenCV (open source computer vision library) is used to realize color recognition of objects after semantic segmentation and calculate their area ratios.

[0074] The visual semantic layer reflects the components of the road environment that the driver visually perceives, such as the sky, roads, protective facilities, traffic signs, etc. Through semantic segmentation technology, components with the same semantic category can be grouped and represented with the same color. Figure 8 As shown, through semantic segmentation technology, feature extraction, feature fusion and semantic segmentation branches are used to form a semantic segmentation network to establish a visual semantic layer. In addition, the visual semantic layer shares the same feature extraction and feature fusion methods with the visual depth layer. First, the input image representing the road environment of the driver's visual perception is processed by a residual network (ResNet) for feature extraction, and then feature fusion is performed through a feature pyramid network (FPN). After the above steps, an image with important features is obtained as the main input of the semantic segmentation branch. In the semantic segmentation branch, the input image is upsampled and convolved to achieve semantic segmentation.

[0075] Six components of the visual road environment with different semantics are extracted in this layer, including road, forest, sky, traffic sign, protective facility, and cliff. In addition, the area ratios of sky, road, protective facility, and forest in the visual road environment are calculated as parameters of the visual semantic layer. The presence of traffic signs and roadside cliffs are also extracted as parameters. These parameters are represented by the area of ​​the sky (represented by AS), the area of ​​the road (AR), the area of ​​protective facilities (APF), the area of ​​the forest (AF), the presence of traffic signs (Sign), and the presence of left or right roadside cliffs (CL and CR).

[0076] Step 4: Based on the depth estimation technology, a visual depth layer is constructed to extract the average depth value in the driver's field of view, reflecting the depth information of the visually perceived road environment (i.e., the driver's perceived distance).

[0077] The visual depth layer reflects the depth information of the road environment perceived by the driver. Figure 8 As shown in FIG. 1 , after using depth estimation (a technique that aims to estimate depth values ​​from images), components of the road environment with different depth values ​​are represented by different colors in the visual depth layer. As the color changes, the depth value (i.e., the driver's perceived distance) of each component of the road visual environment gradually increases. In the present invention, a visual depth layer is constructed by combining different parts, including feature extraction, feature fusion, and depth estimation branches (see FIG. 1 ). Figure 8 ). Convolution and upsampling are applied to the depth estimation branch. From the visual depth layer and the dynamic visual layer, the average depth value (i.e., the average perceived distance) in the driver's field of view is extracted and is referred to as the field of view depth (denoted by DV) in the present invention.

[0078] Step 5: Establish a visual sensitivity layer through the gradient weighted class activation mapping algorithm, and obtain the sensitive distance by calculating the distance from the visual sensitive area to the field of view, thereby reflecting the driver's sensitivity to the entire visual road environment.

[0079] The visual sensitivity layer shows the distribution of the driver's visual sensitivity area in the road environment. Visual sensitivity indicates the degree of attention paid by the driver's eyes to the area, and more attention is paid to the visual sensitivity area representing the area. The present invention establishes a visual sensitivity layer based on the gradient weighted class activation mapping algorithm (Grad-CAM). The overview of the visual sensitivity layer is as follows Fig. 9 As shown. In a convolutional neural network (CNN), a feature is a computer representation of each pixel in an image, and a feature map is a collection of features. After the image is input into a CNN, the gradient information of the last convolutional layer is calculated to obtain a corrected convolutional feature map, since it achieves the best compromise between semantics and detailed spatial information. In order to obtain the visually sensitive area, a global average pooling method is used to assign corresponding weights to the features of the corrected convolutional feature map. All features and their corresponding weights are calculated by a rectified linear unit (ReLU), and the visually sensitive area is visualized in the form of a heat map. The calculation results are shown in equations (5) and (6).

[0080]

[0081]

[0082] Among them, M i represents the i-th feature map, α iis its corresponding weight, ReLU is the rectified linear unit, c1 and c2 are the sizes of the feature maps, Z = c1 × c2, y is the model calculated based on the input image, Represents the feature map M i The pixel value at the jth row and kth column in .

[0083] exist Fig. 9 In the heat map, the red area is the visually sensitive area that the driver pays more attention to. By calculating the distance (pixel) from the visually sensitive area to the field of view of the dynamic visual layer, the sensitive distance (expressed by Dist) is obtained. The sensitive distance reflects the driver's sensitivity to the entire visual road environment perception. In the present invention, the weight of the area with heavier grayscale in the heat map is extracted as the threshold β. When the weight of the feature α_i is greater than β, the feature is regarded as a visually sensitive area in this study, and when the weight of the feature a_id is less than β, it is not involved in the calculation. The sensitive distance is calculated by equations 7 and 8.

[0084] The sensitive distance Dist is the distance from the visual sensitive area to the field of view. The calculation formula of Dist is as follows;

[0085]

[0086]

[0087] Among them, Dist is the sensitive distance; α i is the sensitivity of the region calculated by the gradient weighted class activation mapping algorithm; β is the threshold of the weight, when α i When it is greater than β, the area is a visually sensitive area; ω is the weight of the visually sensitive area of ​​the entire visual road environment; d i is the distance between the ith visual sensitive area and the visual field.

[0088] Step 6: Obtain whether the vehicle is speeding through the driving recorder, and also obtain the driver's age, driving experience and gender parameters.

[0089] Step 7: The parameters extracted before the driver passes the curve are processed by the XGBoost machine learning algorithm to predict whether the driver will illegally occupy the road when passing the curve of the rural two-lane road.

[0090] The present invention uses XGBoost to establish a prediction model for illegal road occupation IROL (illegal road occupation) on curved sections of two-lane rural roads. XGBoost (extreme gradient boosting) is based on the gradient boosting decision tree algorithm and can effectively predict categorical variables. The advantages of XGBoost are its fast speed, high accuracy, and easy control of model complexity when solving data problems. They are achieved by using parallel computing, using second-order Taylor expansion for the objective function, and controlling the regularization term in the objective function. Taking the above advantages into account, the present invention selects XGBoost to establish a prediction model.

[0091] Given a model with m features (x i ∈R^m,y i ∈R), a dataset with n samples and K trees (x i ,y i ), the predicted value of the XGBoost model is calculated by Equation 9.

[0092]

[0093] where h K (x i ) is the Kth decision tree, which is a weak learner; F K (x i ) is a strong learner of K-tree; F K-1 (x i ) is a strong learner of K-1 trees; η is the learning rate.

[0094] The core of XGBoost is to minimize the objective function, which is defined as follows.

[0095]

[0096] in

[0097]

[0098] In the formula is the loss function, is the regularization term, y i is the true value, J is the number of leaves, w tj is the optimal value of the jth leaf node of the tth decision tree, λ is the parameter of the L2 regularization term, and γ is the parameter that controls the number of leaves.

[0099] After applying the second-order Taylor expansion to the objective function, it is transformed into:

[0100]

[0101] Among them G tj is the sum of the first-order derivatives of each leaf node; H tjis the sum of the second-order derivatives of each leaf node. When the derivative of the objective function is 0, w tj The optimal value is -G tj / H tj +λ. Based on the optimal w tj To split the tree. When the node depth of the decision tree reaches the maximum depth of the tree, the splitting stops.

[0102] Step 8: XGBoost is processed through the SHAP interpretable learning algorithm to further analyze how the extracted parameters affect whether illegal lane occupation occurs when turning on a rural two-lane road.

[0103] In order to better explain the results of the prediction model, the present invention applies SHAP (SHapley AdditiveexPlanation). Based on game theory, SHAP proposes an additive explanation model to provide unified explainability for machine learning algorithms. As an explainable machine learning, SHAP solves the black box nature of machine learning, which helps to understand the model output. SHAP can be used in combination with some other machine learning methods such as XGBoost to ensure high prediction accuracy and model explainability. Some commonly used statistical methods, such as logistic regression, can only obtain linear or generalized linear relationships between independent variables and dependent variables by analyzing constant characteristic coefficients. Compared with these methods, SHAP provides more explanations by intuitively displaying linear relationships, nonlinear relationships, interaction effects, and relative importance.

[0104] The SHAP value is defined as the attribution of each feature to the predicted outcome, which is crucial for analyzing and explaining the influencing factors of the dependent variable IROL. Given a model f and a set S with all observed features, the calculation model of the SHAP value is shown in Equation (13).

[0105]

[0106] Among them, ξi is the SHAP value of feature i; N is the set of all features; M is the number of all features.

[0107] The SHAP interaction value represents the contribution of the joint effect of two different variables to the predicted results, showing the interaction effect of the two variables. The SHAP interaction value is calculated using equations (14) and (15).

[0108]

[0109] When i≠j, and

[0110] δ ij (S)=f(S∪{i,j})-f(S∪{i})-f(S∪{j})+f(S) (15)

[0111] Where ξ(i, j) is the SHAP interaction value between feature i and feature j.

[0112] Compared with the traditional lane departure warning technology, the visual road environment model based on the driver's visual perception provides a method for comprehensively quantifying the driver's visual perception of the road environment, providing more information for establishing a high-precision prediction model, making the prediction results more valuable for application; the present invention can be applied to the perception system of autonomous vehicles to enhance its ability to understand the road environment and human driving behavior, which is crucial for the development of human-like autonomous vehicles. In addition, it can also be used in mixed traffic to make traffic more harmonious by reducing the behavioral differences between human driving and autonomous vehicles.

[0113] Embodiment, the prediction method of two-lane rural roads was verified in Tibet, China. The road section studied is a typical two-lane rural road with many bends and low traffic volume. The speed limit on these sections is usually 50km / h, while at sharp turns and steep slopes, the speed limit drops to 20km / h or 30km / h. According to our survey, illegal occupation of the road is one of the main causes of traffic accidents on these sections, accounting for about 21% of total traffic accidents and 25% of fatal traffic accidents.

[0114] like Figure 1 As shown, step 1, based on the driver's visual characteristics, a visual road linear layer is constructed to extract road linear parameters that reflect the driver's perception;

[0115] like Figure 6 As shown, step 2, based on the change of the driver's dynamic field of view with speed, the driver's dynamic visual layer is constructed to lay the foundation for the construction of quantitative driver visual characteristics;

[0116] like Figure 7 As shown, in step 3 and step 4, based on the semantic segmentation technology, a visual semantic layer is constructed to segment the visual road environment, and the area ratios of the sky, roads, protective facilities and forests in the visual road environment, as well as the existence of traffic signs and roadside cliffs are extracted; based on the depth estimation technology, a visual depth layer is constructed to extract the average depth value in the driver's field of view, reflecting the depth information of the visually perceived road environment (i.e., the driver's perceived distance);

[0117] like Figure 8 As shown, step 5, a visual sensitivity layer is established through a gradient weighted class activation mapping algorithm, and the sensitive distance is obtained by calculating the distance from the visual sensitive area to the field of view, thereby reflecting the driver's sensitivity to the entire visual road environment;

[0118] Step 6: Obtain whether the vehicle is speeding through the driving recorder, and also obtain the driver's age, driving experience and gender parameters.

[0119] Step 7: The parameters extracted before the driver passes the curve are processed by the XGBoost machine learning algorithm to predict whether the driver will illegally occupy the road when passing the curve of the rural two-lane road. The prediction results are shown in the following table:

[0120]

[0121] Step 8: Process XGBoost through the SHAP interpretable learning algorithm to further analyze how the extracted parameters affect whether the rural two-lane road will be illegally occupied when turning. The SHAP interpretable analysis overview is shown in the figure below: Figure 5 shown.

[0122] A prediction system for illegal road occupation on a curved road, comprising:

[0123] A visual road alignment module is used to construct a visual road alignment layer based on the driver's visual characteristics, and obtain road alignment parameters that reflect the driver's perception through the visual road alignment layer;

[0124] Driver dynamic vision module: used to construct a driver dynamic vision layer based on the change of the driver's dynamic vision with speed, and obtain the dynamic change of the driver's vision with speed through the driver dynamic vision layer;

[0125] Visual semantic module: used to build a visual semantic layer based on semantic segmentation technology, segment the visual road environment through the visual semantic layer, and thus obtain the components of the road environment visually perceived by the driver;

[0126] Visual depth module: used to construct a visual depth layer based on depth estimation technology, and obtain the depth information of the road environment perceived by the driver through the visual depth layer;

[0127] Visual Sensitivity Distance Module: It is used to construct a visual sensitivity layer through the gradient weighted class activation mapping algorithm, and calculate the distance between the driver's visual sensitivity area and the field of view through the visual sensitivity layer, so as to obtain the driver's visual sensitivity distance;

[0128] Prediction module: used to input road linear parameters, dynamic changes of driver's field of view with speed, components of road environment, depth information of road environment and sensitive distance of driver's vision into the prediction model of two-lane highway curved section IROL constructed by XGBoost machine learning algorithm, output processing results, and predict the road occupation situation of drivers when passing through the curve of rural two-lane road according to the processing results;

[0129] Explanation module: used to process XGBoost through the SHAP interpretable learning algorithm to further analyze whether the obtained parameters affect the illegal occupation of curves on rural two-lane roads.

[0130] The present invention establishes a prediction model with both high accuracy and good interpretability, combining two methods, XGBoost (eXtreme Gradient Boosting) and SHAP (SHapley Additive exPlanation). XGBoost performs well in predicting categorical variables, and SHAP, as an interpretable machine learning method, can better explain the prediction results. According to SHAP's analysis of influencing factors, relevant measures can be taken to reduce the occurrence of illegal road occupation in curved sections of rural roads.

[0131] The above description is only a preferred specific implementation manner of the present invention, but the protection scope of the present invention is not limited thereto. Any technician familiar with the technical field can make equivalent replacements or changes according to the technical scheme and inventive concept of the present invention within the technical scope disclosed by the present invention, which should be covered by the protection scope of the present invention.

Claims

1. A method for predicting illegal road occupation on a curve, characterized in that: The following steps are involved: Based on the visual characteristics of the driver, a visual road alignment layer is constructed, and road alignment parameters reflecting the driver's perception are obtained through the visual road alignment layer; the road alignment parameters reflecting the driver's perception include visual curve curvature and visual curve length; the calculation formulas for the visual curve curvature and visual curve length are: vS iL =S (i+1)L -S iL vS iR =S (i+1)R -S iR Where i = 1, 2, 3, S iL At the control point P iL The cumulative length of the left visual road boundary at , S iR At the control point P iR The cumulative length of the right visual road boundary at iL is the left visual road boundary at control point P iL The tangent angle at iR is the control point P iR The right visual road boundary at vS iL is the control point P iL and P (i+1)L The length of the left visual curve between iL is the control point P iL and P (i+1)L The curvature of the left visual curve between iR is the control point P iR and P (i+1)R The length of the right visual curve between iR is the control point P iR and P (i+1)R The curvature of the right visual curve between; Based on the change of the driver's dynamic field of view with speed, a driver's dynamic visual layer is constructed, and the dynamic change of the driver's field of view with speed is obtained through the driver's dynamic visual layer; Based on the semantic segmentation technology, a visual semantic layer is constructed, and the visual road environment is segmented through the visual semantic layer, so as to obtain the components of the road environment visually perceived by the driver; specifically, the residual network ResNet is used to extract features from the input image representing the road environment visually perceived by the driver, and then the extracted features are fused through the feature pyramid network FPN, and the fused feature image is used as the input of the semantic segmentation branch, and the input feature image is upsampled and convolved, so as to segment the visual road environment; Based on the depth estimation technology, a visual depth layer is constructed, and the depth information of the road environment visually perceived by the driver is obtained through the visual depth layer; the depth information is the driver's perception distance; A visual sensitivity layer is constructed by a gradient weighted class activation mapping algorithm, and the distance between the driver's visual sensitivity area and the field of vision is calculated by the visual sensitivity layer, so as to obtain the driver's visual sensitivity distance; the construction of the visual sensitivity layer comprises the following steps: after inputting the visual road image into a convolutional neural network CNN, a corrected convolution feature map is obtained by calculating the gradient information of the image, a global average pooling method is used to assign corresponding weights to the features of the corrected convolution feature map, all features and their corresponding weights are calculated by a rectified linear unit ReLU, and the visual sensitive area is visualized in the form of a heat map to obtain a visual sensitivity layer; The method of calculating the distance between the driver's visual sensitive area and the visual field through the visual sensitive layer to obtain the driver's visual sensitive distance specifically includes: The calculation process of the driver's visual sensitive area is: Among them, M i represents the i-th feature map, α i is its corresponding weight, ReLU is the rectified linear unit, c1 and c2 are the sizes of the feature maps, Z = c1 × c2, y is the model calculated based on the input image, Represents the feature map M i The pixel value of the jth row and kth column in L Grade-CAM represents the gradient weighted class activation mapping algorithm; The calculation process of the sensitive distance is: Among them, Dist is the sensitive distance; α i is the sensitivity of the region calculated by the gradient weighted class activation mapping algorithm; β is the threshold of the weight, when α i When it is greater than β, the area is a visually sensitive area; ω is the weight of the visually sensitive area of ​​the entire visual road environment; d i is the distance between the ith visual sensitive area and the visual field; The road linear parameters, the dynamic changes of the driver's field of view with speed, the components of the road environment, the depth information of the road environment and the sensitive distance of the driver's vision are input into the prediction model of the IROL of the curved section of the two-lane highway constructed by the XGBoost machine learning algorithm, and the processing results are output. The road occupation situation of the driver when passing through the curve of the rural two-lane road is predicted according to the processing results; the prediction model establishment process of the IROL of the curved section of the two-lane rural highway is as follows: Given a dataset with m features, n samples and K trees (x i ,y i ), x i ∈R^m,y i ∈R; the objective function for calculating the predicted value of the XGBoost model is: where h K (x i ) is the Kth decision tree, F K (x i ) is a strong learner of K-tree, F K-1 (x i ) is a strong learner of K-1 tree, η is the learning rate; Minimize the objective function: In the formula is the loss function, is the regularization term, yi is the true value, J is the number of leaves, and w tj is the optimal value of the jth leaf node of the tth decision tree, λ is the parameter of the L2 regularization term, and γ is the parameter that controls the number of leaves; The second-order Taylor expansion is used for the objective function: Among them G tj is the sum of the first-order derivatives of each leaf node, H tj is the sum of the second-order derivatives of each leaf node. When the derivative of the objective function is 0, w tj The optimal value is -G tj / H tj +λ.

2. A method for predicting illegal road occupation on a curved road according to claim 1, characterized in that: The components of the road environment include: sky, road, protective facilities, traffic signs, forest and roadside cliffs; the depth values ​​of the components of the road environment are respectively represented in the visual depth layer through the depth value estimation technology.

3. The method for predicting illegal road occupation on a curved road according to claim 1, characterized in that: The visual semantic layer and the visual depth layer both adopt feature extraction and feature fusion methods, extract features from an input image representing a driver's visual perception of a road environment through a residual network, and then fuse the extracted features through a feature pyramid network.

4. A prediction system based on the method for predicting illegal road occupation on a curved road according to claim 1, characterized in that: include: A visual road alignment module is used to construct a visual road alignment layer based on the driver's visual characteristics, and obtain road alignment parameters that reflect the driver's perception through the visual road alignment layer; Driver dynamic vision module: used to construct a driver dynamic vision layer based on the change of the driver's dynamic vision with speed, and obtain the dynamic change of the driver's vision with speed through the driver dynamic vision layer; Visual semantic module: used to build a visual semantic layer based on semantic segmentation technology, segment the visual road environment through the visual semantic layer, and thus obtain the components of the road environment visually perceived by the driver; Visual depth module: used to construct a visual depth layer based on depth estimation technology, and obtain the depth information of the road environment perceived by the driver through the visual depth layer; Visual Sensitivity Distance Module: It is used to construct a visual sensitivity layer through the gradient weighted class activation mapping algorithm, and calculate the distance between the driver's visual sensitivity area and the field of view through the visual sensitivity layer, so as to obtain the driver's visual sensitivity distance; Prediction module: It is used to input road linear parameters, dynamic changes of driver's field of view with speed, components of road environment, depth information of road environment and sensitive distance of driver's vision into the prediction model of two-lane highway curved section IROL constructed by XGBoost machine learning algorithm, output the processing results, and predict the road occupation situation of the driver when passing the curve of rural two-lane road based on the processing results.

5. The prediction system for illegal road occupation on curved roads according to claim 4, characterized in that: It also includes an explanation module; the explanation module is used to process XGBoost through the SHAP interpretable learning algorithm, so as to further analyze whether the acquired parameters affect the illegal occupation of the curve of rural two-lane roads.

Images