A guide vehicle driving assistance system based on deep learning

The deep learning-based guided vehicle driving assistance system solves the problems of existing systems being unable to be applied to train driving and having poor accuracy in complex scenarios. It achieves efficient and robust completion of various driving assistance tasks and improves safety during driving.

CN114763136BActive Publication Date: 2024-11-08TONGJI UNIV
View PDF 0 Cites 0 Cited by

Patent Information

Application Number
CN202210289107.7
Authority / Receiving Office
CN · China
Patent Type
Patents(China)
Current Assignee / Owner
Filing Date
2022-03-22
Publication Date
2024-11-08
Estimated Expiration
2042-03-22

AI Technical Summary

Technical Problem

Existing driver assistance systems are often designed for single vehicles and cannot be applied to train driving. Furthermore, the use of traditional non-learning methods results in poor accuracy and robustness in complex and special scenarios, high equipment prices, and insufficient scenario generalization.

Method used

A deep learning-based guided vehicle driving assistance system is adopted, including a forward collision warning subsystem, a driver abnormal behavior detection subsystem, and a lane departure warning subsystem. It utilizes an obstacle detection model, a driver abnormal behavior detection model, and a USFC lane detection module, combined with a ROS reading communication mechanism to achieve data communication.

Benefits of technology

It achieves efficient and robust completion of various driving assistance tasks, and can detect forward collisions, lane departures and driver distraction hazards in advance, thereby improving safety during driving.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN114763136B_ABST
    Figure CN114763136B_ABST
Patent Text Reader

Abstract

The present invention relates to a guide vehicle driving assistance system based on deep learning, which includes: a forward collision warning subsystem: detecting forward obstacles, and estimating the distance between the forward obstacles and the guide vehicle to perform collision judgment and warning; a driver abnormal behavior detection subsystem: establishing a driver abnormal behavior detection model based on contrastive learning to determine whether the driver has abnormal driving behavior; a lane line deviation warning subsystem: detecting lane lines through a USFC lane line detection module, and determining whether lane line deviation occurs based on relative position. Compared with the prior art, the present invention has a low deployment price, strong system generalization, and a wide coverage. It can efficiently and robustly complete driving assistance tasks, and can be conveniently and massively deployed in existing driving systems, so that the intelligent vehicle train guide vehicle can detect the danger of forward collision and lane deviation in advance and avoid driver distraction, and respond and warn in time, thereby improving the safety during driving.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present invention relates to the field of intelligent driving technology, and in particular to a guide vehicle driving assistance system based on deep learning. Background Art

[0002] Active safety technology has attracted much attention because it can enable drivers to detect possible dangers as quickly as possible, thereby attracting attention and improving safety. At the same time, forward collision warning, abnormal driver behavior detection and lane departure warning, as the most common tasks of driver assistance systems, have also become one of the current research hotspots in autonomous driving: forward collision refers to a vehicle that is too close to the vehicle in front during driving, which may cause a traffic accident; abnormal driver behavior refers to the driver's behaviors that hinder normal driving, such as smoking, making phone calls, and dozing off, which will bring potential traffic hazards; lane departure refers to the vehicle's driving behavior that deviates from its own driving lane and leans towards the adjacent lane during driving, thus creating traffic safety hazards.

[0003] In recent years, with the investment of major manufacturers in driver assistance systems, many driver assistance systems have emerged. These auxiliary algorithms have excellent detection effects and performance on open object detection datasets, but the existing systems have the following shortcomings:

[0004] Firstly, existing driver assistance systems are often designed for single vehicles, which makes them unsuitable for vehicle train driving. For example, they often do not integrate multiple functions, which creates certain traffic hazards.

[0005] Secondly, existing driver assistance algorithms often use traditional non-learning methods such as expert models, resulting in poor accuracy and robustness when facing complex scenarios and some special situations.

[0006] Third, a small number of existing algorithms based on deep learning still have shortcomings in terms of equipment price and scenario generalization. For example, distance estimation for forward collision warning often relies on expensive lidar or complex binocular systems, and abnormal behavior monitoring relies on classification networks that have a high reliance on data sets. Summary of the invention

[0007] The purpose of the present invention is to provide a deep learning-based driving assistance system for a guided vehicle in order to overcome the defects of the above-mentioned prior art.

[0008] The purpose of the present invention can be achieved by the following technical solutions:

[0009] A deep learning-based driving assistance system for a guided vehicle, the system comprising:

[0010] Forward collision warning subsystem: detects forward obstacles through the obstacle detection model, and establishes camera pose estimation to predict the distance between the forward obstacle and the guide vehicle for collision judgment and warning;

[0011] Driver abnormal behavior detection subsystem: establishes a driver abnormal behavior detection model based on contrastive learning to determine whether the driver has abnormal driving behavior;

[0012] Lane Departure Warning Subsystem: The lane line is detected through the USFC lane line detection module, and a multi-threshold TLC model algorithm is established to determine whether a lane line deviation occurs based on the relative position;

[0013] ROS reading communication mechanism subsystem: includes video stream output node, network data reading node and network output node, which are used to realize data communication of each subsystem.

[0014] The forward collision warning subsystem includes the following steps:

[0015] 11) Install a camera on the guide vehicle to obtain forward-looking video, and calibrate the internal and external parameters of the camera;

[0016] 12) Build and train an obstacle detection model to detect obstacles including vehicles and pedestrians in traffic scenes;

[0017] 13) Establish a camera geometric imaging model to obtain detection information from the obstacle detection model and estimate the distance between the guide vehicle and the forward obstacle;

[0018] 14) When the distance between the guide vehicle and the forward obstacle is lower than the distance threshold, an alarm is issued.

[0019] In the step 12), the obstacle detection model is specifically a target detection network based on YOLOv5.

[0020] When the obstacle detection model is trained, the loss function is expressed as:

[0021]

[0022] Among them, λ coord , noobj are hyperparameters, x i is the position of the center of the prediction box on grid i on the x-axis, is the position of the center of the real box on grid i on the x-axis, y i is the position of the center of the prediction box on grid i on the y-axis, is the position of the center of the real box on the grid i on the y-axis, S 2Indicates that a picture is divided into S×S block grids, B is the number of prediction boxes in each grid, and w i is the width of the prediction box on grid i, is the width of the real box on grid i, h i is the height of the prediction box on grid i, is the height of the real box on grid i, C i is the confidence of the prediction box on grid i, is the confidence of the ground-truth box of grid i, To determine whether the j-th predicted box in the i-th grid corresponds to the real box, if so, it is 1, otherwise it is 0. To determine whether the j-th predicted box in the i-th grid does not correspond to the real box, if so, it is 1, otherwise it is 0, p i (c) is the probability prediction of whether the grid i is of the cth category, is the true value of the cth class on grid i, Localization error is the coordinate prediction loss, Confidence error is the confidence prediction loss, and Class probabilities error is the category prediction loss.

[0023] The hyperparameter λ coord The value is 5 to give more emphasis to the position detection loss. The hyperparameter λ noobj A value of 0.5 reduces the focus on position confidence for meshes without objects.

[0024] In the step 13), the distance between the guide vehicle and the forward obstacle is obtained through the camera geometric imaging model, specifically:

[0025] Assuming that the guide vehicle is traveling on a horizontal road, the projection model is an ideal geometric model, and the camera is parallel to the ground. The distance Z between the guide vehicle and the forward obstacle is calculated by combining the geometric coordinates of the road surface on which the guide vehicle is traveling and the guide vehicle body with the internal parameters calibrated by the camera, then:

[0026] Z=f*H / y=F*H / Y

[0027] Among them, f is the focal length of the camera, y is the distance between the contact point of the forward obstacle and the road surface and the center line of the image, H is the distance between the camera and the ground, F is the equivalent vertical focal length of the camera, and Y is the number of pixels from the bottom of the guide vehicle to the middle of the image.

[0028] The driver abnormal behavior detection subsystem performs driver abnormal behavior detection including the following steps:

[0029] 21) Install a camera on the guide vehicle to obtain the cab video, and calibrate the internal and external parameters of the camera;

[0030] 22) Build and train a driver abnormal behavior detection model;

[0031] 23) Perform abnormal behavior detection using the trained driver abnormal behavior detection model, specifically:

[0032] Based on contrastive learning, the image behavior feature vector extracted by the abnormal behavior detection model is compared with the feature vector of normal driving behavior. If the vector difference is less than the abnormal threshold, it is judged as normal driving behavior, otherwise it is abnormal driving behavior.

[0033] In the step 22), the driver abnormal behavior detection model includes:

[0034] Video stream feature extraction module based on 3D-CNN: used to extract the features of the cab video image stream and obtain a 512-dimensional feature vector;

[0035] Two-layer fully connected network module: used to compress and interact with high-dimensional features of video image streams, input 512-dimensional feature vectors, and obtain 128-dimensional feature vectors;

[0036] L2 regularized network module: used to normalize the feature vector so that it can be compared. A 128-dimensional feature vector is input to obtain a regularized 128-dimensional feature vector.

[0037] In the process of training the driver abnormal behavior detection model, the model maximizes the normal driving behavior v ny and v nx similarity between them, while minimizing the normal driving behavior v ny and abnormal driving behavior am The similarity between is taken as the goal, and the optimization iteration is performed, then the loss function The expression is:

[0038]

[0039] in, is the indicator function, which returns 1 when x≠y and 0 in other cases, τ is the temperature coefficient, K is the number of normal behaviors, M is the number of abnormal behaviors, and L xy Represents the minimum loss function of a pair of positive pairs of positive sample x and all its negative pairs.

[0040] The lane departure warning subsystem performs lane detection through a lane detection model based on USFC, obtains lane detection information through the input forward-looking video image, and issues a lane departure warning. When one of the following conditions is met, the identifier value is 1 and a warning is issued:

[0041]

[0042] u=θ>θ0

[0043] u=L / v <t0

[0044] Among them, u is an identifier for determining whether to issue a warning. If u=1, a warning is issued, and if u=0, no warning is issued. d1 is the distance between the left lane line of the current vehicle and the center of the vehicle, d2 is the distance between the right lane line of the current vehicle and the center of the vehicle, W is the vehicle width, D0 is the set vehicle deviation threshold, θ is the lane line deviation angle, θ0 is the corresponding lane line deviation angle threshold, t0 is the corresponding threshold time for exiting the lane line, L is the distance to completely exit the current lane, and v is the current vehicle speed.

[0045] Compared with the prior art, the present invention has the following advantages:

[0046] The present invention proposes a driving assistance system to address the problems that current driver assistance systems are often limited to a single vehicle and existing non-learning methods. The system can efficiently and robustly complete driving assistance tasks and can be conveniently deployed in existing intelligent driving systems, so that the intelligent vehicle train guide vehicle can detect the danger of front collision and lane departure in advance and avoid driver distraction, and respond and warn in time to improve safety during driving. BRIEF DESCRIPTION OF THE DRAWINGS

[0047] Figure 1 It is a schematic diagram of the front impact subsystem in the present invention.

[0048] Figure 2 Schematic diagram of the driver abnormal behavior detection subsystem in the present invention.

[0049] Figure 3 Schematic diagram of the lane departure warning subsystem in the present invention.

[0050] Figure 4 The figure is a schematic diagram of the structure of the YOLO module in the forward collision warning subsystem of the present invention.

[0051] Figure 5 Schematic diagram of the forward obstacle distance estimation module in the forward collision warning subsystem of the present invention

[0052] Figure 6 It is a schematic diagram of the 3D-CNN structure in the driver abnormal behavior detection subsystem in the present invention.

[0053] Figure 7 This is a structural diagram of the fully connected layer and L2 regularization in the driver abnormal behavior detection subsystem in the present invention.

[0054] Figure 8This is a structural diagram of the USFC in the lane departure warning subsystem of the present invention.

[0055] Fig. 9 This is a flow chart of the lane departure warning algorithm in the lane departure warning subsystem of the present invention. DETAILED DESCRIPTION

[0056] The present invention is described in detail below with reference to the accompanying drawings and specific embodiments.

[0057] Example

[0058] The present invention provides a guide vehicle driving assistance system based on deep learning, which includes a forward collision warning subsystem, a driver abnormal behavior detection subsystem and a lane line deviation warning subsystem. In the forward collision warning subsystem, an obstacle warning model based on YOLO-V5 is used to achieve fast and efficient detection of forward obstacles, and then the distance between the forward obstacle and the vehicle is estimated based on the camera geometric imaging model; the driver abnormal behavior detection subsystem is based on contrast learning, and detects whether it is an abnormal behavior by judging whether the feature vector offset is greater than a threshold γ. In the lane line deviation warning subsystem, the USFC algorithm is used to achieve efficient, robust and accurate detection of lane lines, and whether deviation occurs is judged by relative position.

[0059] The steps to build the above-mentioned guide vehicle driving assistance system are as follows:

[0060] 1) Establish a forward collision warning subsystem. The specific steps are as follows:

[0061] (1-1) A camera for obtaining forward-looking video is installed on the guide vehicle, and the internal and external parameters of the camera are calibrated. The internal parameters are used to eliminate image distortion caused by the lens, and the external parameters are used to determine the corresponding position of the points on the image in the real world. The calibration process can refer to the chessboard method. Use the camera to be calibrated to take multiple images containing the chessboard calibration plate, and then input them into the Calibration toolbox of Matlab to obtain the internal and external parameters of the camera;

[0062] (1-2) Construct an obstacle warning model and use the constructed data for training to obtain ideal network parameters, specifically:

[0063] First, a target detection network based on YOLOv5 is established to detect objects in the input forward-looking video image information and obtain the detection frame information of the forward obstacles.

[0064] Then, a distance detection module based on the camera geometric imaging model is established, and the obstacle detection information in the previous step is input into the module to estimate the distance between the obstacle and the vehicle. The process is as follows:

[0065] The distance between the guide vehicle and the front obstacle is calculated by combining the geometric coordinates of the road surface on which the guide vehicle is traveling and the guide vehicle body with the internal parameters calibrated by the camera. Assuming that the guide vehicle is traveling on a horizontal road, the projection model is an ideal geometric model, and the camera is parallel to the ground. Let point p represent the camera, which is very small relative to the volume of the vehicle and can be regarded as a mass point. f represents the focal length of the camera, H is the distance between the camera and the ground, y is the distance between the contact point of the front obstacle B and the road surface and the midline of the image, and Z is assumed to be the actual distance between the guide vehicle A and the front obstacle B. According to the triangle similarity principle, it can be obtained that: y / f=H / Z. Therefore, the actual distance between the guide vehicle and the front obstacle can be obtained as:

[0066] Z=f*H / y

[0067] After the camera parameters are set, the calculation formula for the actual distance between the guide vehicle and the forward obstacle can be written as follows:

[0068] Z=f*H / y=F*H / Y

[0069] Where F is the equivalent vertical focal length of the camera, and Y is the number of pixels from the bottom of the vehicle to the middle of the image.

[0070] Finally, the data set is input into the YOLO v5 network, the YOLO v5 network is trained, and the network parameters are updated using the loss function back propagation to obtain the final result. The loss function is:

[0071]

[0072] Among them, the hyperparameter λ coord =5, more emphasis is placed on position detection loss; noobj = 0.5, for grids without objects, reduce the focus on their position confidence,,x i is the position of the center of the prediction box on grid i on the x-axis, is the position of the center of the real box on grid i on the x-axis, y i is the position of the center of the prediction box on grid i on the y-axis, is the position of the center of the real box on the grid i on the y-axis, S 2 A picture is divided into S×S block grids, B is the number of prediction boxes in each grid, and w i is the width of the prediction box on grid i, is the width of the real box on grid i, h i is the height of the prediction box on grid i, is the height of the real box on grid i, C i is the confidence of the prediction box on grid i, is the confidence of the ground-truth box of grid i, To determine whether the j-th predicted box in the i-th grid corresponds to the real box, if so, it is 1, otherwise it is 0. To determine whether the j-th predicted box in the i-th grid does not correspond to the real box, if so, it is 1, otherwise it is 0, p i (c) is the probability prediction of whether the grid i is of the cth category, is the true value of the cth class on grid i, Localization error is the coordinate prediction loss, Confidence error is the confidence prediction loss, and Class probabilities error is the category prediction loss.

[0073] (1-3) Build a ROS reading communication mechanism to provide forward obstacle warning for real-time video information. The ROS communication mechanism includes video stream output node, network data reading node and network output node.

[0074] 2) Establish a driver abnormal behavior detection subsystem. The specific steps are as follows:

[0075] (2-1) A camera is installed on the guide vehicle to obtain the cab video, and the internal and external parameters of the camera are calibrated. The internal parameters are used to eliminate the image distortion caused by the lens, and the external parameters are used to determine the corresponding position of the points on the image in the real world. The calibration process can refer to the chessboard method. Use the camera to be calibrated to take multiple images containing the chessboard calibration plate, and then input them into the Calibration toolbox of Matlab to obtain the internal and external parameters of the camera;

[0076] (2-2) Construct a driver abnormal behavior detection model, specifically:

[0077] Firstly, a video stream feature extraction module based on 3D-CNN is established to extract features from the input cab video image stream and obtain a 512-dimensional feature vector.

[0078] Then, a two-layer fully connected network module is established, and the 512-dimensional feature vector is input into it to obtain a 128-dimensional feature vector;

[0079] Afterwards, an L2 regularized network module is established, and the 128-dimensional feature vector is input into it to obtain a regularized 128-dimensional vector, which is compared with the feature vector of normal driving behavior obtained through training. If the difference is less than a certain threshold, it is normal driving behavior, otherwise it is abnormal driving behavior.

[0080] In the process of training the driver abnormal behavior detection model, assume that there are K normal behaviors and M abnormal behaviors, label them as z∈1,…,K+M, and label the output feature vector of the xth pair of normal and abnormal behaviors as vnx and v ax , so for each normal sample, we can select K(K-1) positive pairs and KM negative pairs, then the loss function The expression is:

[0081]

[0082] in, is the indicator function, which returns 1 when j≠i and 0 in other cases. τ is the temperature coefficient, L xy Represents the minimum loss function of a pair of positive pairs and all its negative pairs of positive samples x. Since the feature vector v has been normalized, Represents cosine similarity. The model maximizes the normal driving behavior v ny and v nx The similarity between them, while minimizing the normal driving behavior v ny and abnormal driving behavior am The similarity between them is taken as the goal and optimization iterations are performed.

[0083] (2-3) Build a ROS reading communication mechanism to detect abnormal driver behavior based on real-time video information. The ROS communication mechanism includes video stream output nodes, network data reading nodes, and network output nodes.

[0084] 3) Establish a lane deviation subsystem. The specific steps are as follows:

[0085] (3-1) A camera for obtaining forward-looking video is installed on the guide vehicle, and the internal and external parameters of the camera are calibrated. The internal parameters are used to eliminate image distortion caused by the lens, and the external parameters are used to determine the corresponding position of the points on the image in the real world. The calibration process can refer to the chessboard method. Use the camera to be calibrated to take multiple images containing the chessboard calibration plate, and then input them into the Calibration toolbox of Matlab to obtain the internal and external parameters of the camera;

[0086] (3-2) Construct a lane deviation model and use the constructed data for training to obtain ideal network parameters, specifically:

[0087] First, a lane detection network based on USFC is established to detect lanes based on the image information of the input forward-looking video to obtain lane detection information. This is different from the existing lane detection algorithm, which models the problem as a full-pixel segmentation problem. The lane detection problem is modeled as a set of positions of lanes in certain rows in the image, that is, based on position selection and classification in the row direction, only h artificially set row classification problems need to be processed, and the classification problem of each row is w-dimensional.

[0088] (3-3) Establish a TLC model with comprehensive multi-scenario thresholds. Input the lane line detection information in the previous step into the module. The module uses relative position to perform lane line warning. According to different road scenarios, a comprehensive multi-scenario threshold is selected to ensure that the entire deviation warning is more accurate. Then:

[0089] u=d1-W / 2 <D0;or u=d2-W / 2<D0;oru=θ> θ0;oru=L / v <t0。

[0090] Among them, u is an identifier for determining whether to issue a warning. If u=1, a warning is issued, and if u=0, no warning is issued. d1 is the distance between the left lane line of the current vehicle and the center of the vehicle, d2 is the distance between the right lane line of the current vehicle and the center of the vehicle, W is the vehicle width, D0 is the set vehicle deviation threshold, θ is the lane line deviation angle, θ0 is the corresponding lane line deviation angle threshold, t0 is the corresponding threshold time for exiting the lane line, L is the distance to completely exit the current lane, and v is the current vehicle speed.

[0091] (3-4) Build a ROS reading communication mechanism to provide lane deviation warning for real-time video information. The ROS communication mechanism includes video stream output node, network data reading node and network output node.

[0092] In summary, the present invention proposes a guided logistics vehicle guide vehicle driving assistance system based on deep learning, which can complete driving assistance tasks efficiently and robustly, and can be conveniently deployed in the existing intelligent driving system, so that the intelligent vehicle train following vehicle can detect the danger of front collision and lane departure in advance and avoid driver distraction, and respond and warn in time, thereby improving the safety during driving.

[0093] It is obvious that those skilled in the art can easily make various modifications to these embodiments and apply the general principles described herein to other embodiments without creative work. Therefore, the present invention is not limited to the embodiments herein, and improvements and modifications made by those skilled in the art based on the disclosure of the present invention without departing from the scope of the present invention should be within the protection scope of the present invention.

Claims

1. A deep learning-based driving assistance system for a guided vehicle, characterized in that: The system includes: Forward collision warning subsystem: detects forward obstacles through the obstacle detection model, and establishes camera pose estimation to predict the distance between the forward obstacle and the guide vehicle for collision judgment and warning; Driver abnormal behavior detection subsystem: establishes a driver abnormal behavior detection model based on contrastive learning to determine whether the driver has abnormal driving behavior; Lane Departure Warning Subsystem: The lane line is detected through the USFC lane line detection module, and a multi-threshold TLC model algorithm is established to determine whether a lane line deviation occurs based on the relative position; ROS reading communication mechanism subsystem: including video stream output node, network data reading node and network output node, which are used to realize data communication of each subsystem; The forward collision warning subsystem includes the following steps: 11) Install a camera on the guide vehicle to obtain forward-looking video, and calibrate the internal and external parameters of the camera; 12) Build and train an obstacle detection model to detect obstacles including vehicles and pedestrians in traffic scenes; 13) Establish a camera geometric imaging model to obtain detection information from the obstacle detection model and estimate the distance between the guide vehicle and the forward obstacle; 14) When the distance between the guide vehicle and the forward obstacle is lower than the distance threshold, an alarm is issued; In the step 12), the obstacle detection model is specifically a target detection network based on YOLOv5; When the obstacle detection model is trained, the loss function is expressed as: Among them, λ coord , noobj are hyperparameters, x i is the position of the center of the prediction box on grid i on the x-axis, is the position of the center of the real box on grid i on the x-axis, y i is the position of the center of the prediction box on grid i on the y-axis, is the position of the center of the real box on the grid i on the y-axis, S 2 Indicates that a picture is divided into S×S block grids, B is the number of prediction boxes in each grid, and w i is the width of the prediction box on grid i, is the width of the real box on grid i, h i is the height of the prediction box on grid i, is the height of the real box on grid i, C i is the confidence of the prediction box on grid i, is the confidence of the ground-truth box of grid i, To determine whether the j-th predicted box in the i-th grid corresponds to the real box, if so, it is 1, otherwise it is 0. To determine whether the j-th predicted box in the i-th grid does not correspond to the real box, if so, it is 1, otherwise it is 0, p i (c) is the probability prediction of whether the grid i is of the cth category, is the true value of the cth class on grid i, Localization error is the coordinate prediction loss, Confidence error is the confidence prediction loss, and Class probabilities error is the category prediction loss.

2. A deep learning-based guided vehicle driving assistance system according to claim 1, characterized in that: The hyperparameter λ coord The value is 5 to give more emphasis to the position detection loss. The hyperparameter λ noobj A value of 0.5 reduces the focus on position confidence for meshes without objects.

3. The deep learning-based guided vehicle driving assistance system according to claim 1, characterized in that: In the step 13), the distance between the guide vehicle and the forward obstacle is obtained through the camera geometric imaging model, specifically: Assuming that the guide vehicle is traveling on a horizontal road, the projection model is an ideal geometric model, and the camera is parallel to the ground. The distance Z between the guide vehicle and the forward obstacle is calculated by combining the geometric coordinates of the road surface on which the guide vehicle is traveling and the guide vehicle body with the internal parameters calibrated by the camera, then: Z=f*H / y=F*H / Y Among them, f is the focal length of the camera, y is the distance between the contact point of the forward obstacle and the road surface and the center line of the image, H is the distance between the camera and the ground, F is the equivalent vertical focal length of the camera, and Y is the number of pixels from the bottom of the guide vehicle to the middle of the image.

4. The deep learning-based guided vehicle driving assistance system according to claim 1, characterized in that: The driver abnormal behavior detection subsystem performs driver abnormal behavior detection including the following steps: 21) Install a camera on the guide vehicle to obtain the cab video, and calibrate the internal and external parameters of the camera; 22) Build and train a driver abnormal behavior detection model; 23) Perform abnormal behavior detection using the trained driver abnormal behavior detection model, specifically: Based on contrastive learning, the image behavior feature vector extracted by the abnormal behavior detection model is compared with the feature vector of normal driving behavior. If the vector difference is less than the abnormal threshold, it is judged as normal driving behavior, otherwise it is abnormal driving behavior.

5. The deep learning-based guided vehicle driving assistance system according to claim 4, characterized in that: In the step 22), the driver abnormal behavior detection model includes: Video stream feature extraction module based on 3D-CNN: used to extract the features of the cab video image stream and obtain a 512-dimensional feature vector; Two-layer fully connected network module: used to compress and interact with high-dimensional features of video image streams, input 512-dimensional feature vectors, and obtain 128-dimensional feature vectors; L2 regularized network module: used to normalize the feature vector so that it can be compared. A 128-dimensional feature vector is input to obtain a regularized 128-dimensional feature vector.

6. A deep learning-based guided vehicle driving assistance system according to claim 5, characterized in that: In the process of training the driver abnormal behavior detection model, the model maximizes the normal driving behavior v ny and v nx similarity between them, while minimizing the normal driving behavior v ny and abnormal driving behavior am The similarity between is taken as the goal, and the optimization iteration is performed, then the loss function The expression is: in, is the indicator function, which returns 1 when x≠y and 0 in other cases, τ is the temperature coefficient, K is the number of normal behaviors, and M is the number of abnormal behaviors. Represents the minimum loss function of a pair of positive pairs of positive sample x and all its negative pairs.

7. The deep learning-based guided vehicle driving assistance system according to claim 1, characterized in that: The lane departure warning subsystem performs lane detection through a lane detection model based on USFC, obtains lane detection information through the input forward-looking video image, and issues a lane departure warning. When one of the following conditions is met, the identifier value is 1 and a warning is issued: u=θ>θ0 u=L / v <t0 Among them, u is an identifier for determining whether to issue a warning. If u=1, a warning is issued, and if u=0, no warning is issued. d1 is the distance between the left lane line of the current vehicle and the center of the vehicle, d2 is the distance between the right lane line of the current vehicle and the center of the vehicle, W is the vehicle width, D0 is the set vehicle deviation threshold, θ is the lane line deviation angle, θ0 is the corresponding lane line deviation angle threshold, t0 is the corresponding threshold time for exiting the lane line, L is the distance to completely exit the current lane, and v is the current vehicle speed.