Dangerous behavior early warning method based on driver visual feature fusion

Abstract

The invention relates to the technical field of visual feature perception, in particular to a dangerous behavior early warning method based on driver visual feature fusion. Comprising the steps that a multi-modal data packet is generated according to collected multi-source image data, and the multi-modal data packet comprises first-level modal data, second-level modal data and third-level modal data; processing all data in the multi-modal data packet according to a preset fusion model, and generating a fusion feature image according to a fusion result; and establishing a visual perception model, and generating a driver visual perception result according to the visual perception model and the fused feature image. According to the method, the time sequence is established, the acquired image data in the driving process of the driver is continuously acquired, and the image data in the driving process of the driver is accurately processed according to the preset visual processing model, so that the sight direction of the driver and the accurate prediction of dangerous driving behaviors are improved; therefore, the early warning efficiency of potential driving risks is improved.

Description

Technical Field

[0001] The present application relates to the field of visual feature perception technology, and in particular to a dangerous behavior warning method based on the fusion of driver visual features. Background Art

[0002] With the application of artificial intelligence in smart transportation and autonomous driving, the construction of predictive models of driver driving behavior has become an important support for achieving traffic safety and efficient driving. These predictive models can better understand and predict the driver's behavior during driving, thereby improving the performance and safety of the autonomous driving system.

[0003] However, the reference indicators of the current prediction models for driver driving behavior are often the external environmental parameters of the vehicle and the operating parameters of the vehicle itself. There is no model for predicting the driver's line of sight and driving behavior in the driving scene. As the largest participant in the traffic system, the driver is the dominant factor affecting road traffic safety. 90% of the road traffic information required during driving comes from vision. Summary of the invention

[0004] The purpose of this application is: to solve the above-mentioned technical problems, this application provides a dangerous behavior warning method based on the fusion of driver's visual features, aiming to improve the processing efficiency and accuracy of the driver's visual features and improve the warning efficiency of the driver's dangerous behavior.

[0005] In some embodiments of the present application, based on a visual multimodal feature fusion network, corresponding feature preprocessing feature models are established according to different visual modalities to process feature data of three modalities, and the complementarity between different modalities is used to correct the feature data of different modalities. By embedding a cross-modal feature fusion module, deep separable convolution is used to fuse the feature data of different modalities to generate a fused feature map, thereby realizing information fusion between different modalities and improving the accuracy of understanding visual information.

[0006] In some embodiments of the present application, by establishing a time series, the image data collected during the driver's driving process is continuously collected, and according to a preset visual processing model, the image data during the driver's driving process is accurately processed to improve the accurate prediction of the driver's line of sight and dangerous driving behavior, thereby improving the efficiency of early warning of potential driving risks.

[0007] In some embodiments of the present application, a dangerous behavior warning method based on driver visual feature fusion is provided, comprising:

[0008] Generate a multimodal data packet according to the collected multi-source image data, wherein the multimodal data packet includes: primary modal data, secondary modal data and tertiary modal data;

[0009] Process all data in the multimodal data packet according to the preset fusion model, and generate a fusion feature image according to the fusion result;

[0010] Establishing a visual perception model, and generating the driver's visual perception results based on the visual perception model and the fused feature image;

[0011] Generate corresponding warning instructions based on the driver's visual perception results.

[0012] In some embodiments of the present application, the generating of the multimodal data packet includes:

[0013] Acquire multi-source image data;

[0014] Based on the time series, establish the time sequence T, T = (t1, t2...t m ), where t i is the i-th moment, m is the number of moments;

[0015] Set t in sequence i For the target moment;

[0016] Extract the RGB image features at the target moment in the multi-source image data according to the preprocessing feature model, and generate the primary modality data according to the extraction results;

[0017] Extract infrared features at the target time in multi-source image data according to the preprocessing feature model, and generate secondary modal data according to the extraction results;

[0018] The deep features of the target moment in the multi-source image data are extracted according to the preprocessing feature model, and the tertiary modal data are generated according to the extraction results.

[0019] In some embodiments of the present application, when generating a fused feature image according to the fusion result, the method includes:

[0020] Establishing a feature correction model;

[0021] Correcting the primary modal data, the secondary modal data, and the tertiary modal data according to the feature correction model;

[0022] Generate first-level features to be fused, second-level features to be fused, and third-level features to be fused according to the correction results;

[0023] According to the deep separable convolution technology, the first-level features to be fused, the second-level features to be fused, and the third-level features to be fused are linearly embedded and fused;

[0024] Generate a fused feature image at the target moment;

[0025] Generate the fused feature images at each moment in sequence, and establish a sequence B of fused feature images, B = (b1, b2... b m ); where b i is the fused feature image at the i-th moment.

[0026] In some embodiments of the present application, when establishing the visual perception model, it includes:

[0027] Based on historical parameters, establish a plurality of sample training data packets, and establish a sequence W of sample training data packets, W = (w1, w2... w n ), where w i is the i-th sample training data packet; n is the number of sample training data packets;

[0028] Select w i in sequence as the target sample training data packet;

[0029] Generate a sequence I of face images according to the target sample training data packet, I = (I1, I2... I t );

[0030] The sequence I of face images generates a feature mapping sequence after feature extraction;

[0031] Construct a primary processing model according to the feature mapping sequence;

[0032] Generate a loss evaluation value of the primary processing model, and judge whether to generate a correction instruction according to the loss evaluation value;

[0033] Output the line-of-sight prediction sub-model.

[0034] In some embodiments of the present application, when judging whether to generate a correction instruction, it includes:

[0035] Preset a first loss evaluation value threshold F1 and a second loss evaluation value threshold F2;

[0036] If f < F1, set the current primary processing model as the line-of-sight prediction sub-model;

[0037] If F1 ≤ f < F2, generate a primary correction instruction, and set the next sample training data packet as the target sample data packet according to the primary correction instruction, and perform iterative optimization on the current primary processing model;

[0038] If f ≥ F2, generate a secondary correction instruction, update the sequence of sample training data packets according to the secondary correction instruction, re-select the target sample data packet according to the update result, and construct a primary processing model.

[0039] In some embodiments of the present application, when establishing the visual perception model, it further includes:

[0040] Generate multiple dangerous behavior data packets based on historical parameters;

[0041] Constructing a behavior prediction sub-model based on all dangerous behavior data packets;

[0042] A visual perception model is constructed based on the behavior recognition sub-model and the sight line prediction sub-model.

[0043] In some embodiments of the present application, when generating the driver's visual perception result, it includes:

[0044] Obtain fused feature image sequence B;

[0045] Generate the sight deviation degree sequence H according to the fused feature image sequence B and the sight prediction sub-model, H = (h1, h2...h m ), where h i is the line of sight deviation at the i-th moment;

[0046] Generate a first reference evaluation value J1 according to the sight deviation degree column H;

[0047]

[0048] Among them, e1 is the preset first weight coefficient, e2 is the preset second weight coefficient, Q1 is the preset first fixed coefficient, Q2 is the preset second fixed coefficient, h' is the safety deviation, Y(i) is the first selection coefficient, if h i -h'≤0, Y(i)=0, if h i -h'>0, Y(i)=1, ΔT is the time interval between adjacent moments.

[0049] In some embodiments of the present application, when generating the driver's visual perception result, it also includes:

[0050] Acquire fused feature image data B;

[0051] Generate a behavior risk assessment value sequence D based on the fused feature image sequence B and the behavior recognition sub-model, D = (d1, d2…d m ), where d i is the behavior risk assessment value at moment i;

[0052] Generate a second reference evaluation value J2 based on the behavior risk evaluation value sequence D

[0053]

[0054] Among them, e3 is the preset third weight coefficient, e4 is the preset fourth weight coefficient, Q3 is the preset third fixed coefficient, Q4 is the preset fourth fixed coefficient, d' is the safety behavior value, P(i) is the second selection coefficient, if d i-d' ≤ 0, P(i) = 0, if d i -d' > 0, P(i) = 1, where ΔT is the time interval between adjacent moments.

[0055] In some embodiments of the present application, corresponding warning instructions are generated according to the driver's visual perception results, including:

[0056] Generate a risk evaluation value k according to the first reference evaluation value J1 and the second reference evaluation value J2;

[0057] k = e5 * J1 + e6 * J2;

[0058] where e5 is a preset fifth weight coefficient and e6 is a preset sixth weight coefficient;

[0059] Set the risk evaluation value K as the driver's visual perception result, and preset the first risk evaluation value threshold K1 and the second risk evaluation value threshold K2;

[0060] If k ≤ K1, no warning instruction is generated;

[0061] If K1 < k ≤ K2, a first-level warning instruction is generated;

[0062] If k > K2, a second-level warning instruction is generated.

[0063] Compared with the prior art, the beneficial effects of a method for warning of dangerous behaviors based on driver visual feature fusion in an embodiment of the present application are as follows:

[0064] Based on a visual multi-modal feature fusion network, corresponding feature preprocessing feature models are established according to different visual modalities, so as to process the feature data of three modalities, correct different modality feature data by using the complementarity between different modalities, and through embedding a cross-modal feature fusion module, use depthwise separable convolution to fuse the feature data of different modalities to generate a fused feature map, realizing the information fusion between different modalities and improving the accuracy of visual information understanding.

[0065] By establishing a time series, continuously collect the image data during the driver's driving process, and according to a preset visual processing model, realize precise processing of the image data during the driver's driving process, improve the precise prediction of the driver's line of sight direction and dangerous driving behaviors, and thus improve the warning efficiency for potential driving risks. BRIEF DESCRIPTION OF THE DRAWINGS

[0066] Figure 1 is a schematic flowchart of a method for warning of dangerous behaviors based on driver visual feature fusion in a preferred embodiment of an embodiment of the present application. DETAILED DESCRIPTION

[0067] The specific implementation methods of the present application are further described in detail below in conjunction with the accompanying drawings and examples. The following examples are used to illustrate the present application but are not intended to limit the scope of the present application.

[0068] In the description of the present application, it should be understood that the terms "center", "up", "down", "front", "back", "left", "right", "vertical", "horizontal", "top", "bottom", "inside", "outside", etc., indicating orientations or positional relationships, are based on the orientations or positional relationships shown in the accompanying drawings, and are only for the convenience of describing the present application and simplifying the description, and do not indicate or imply that the device or element referred to must have a specific orientation, be constructed and operated in a specific orientation, and therefore should not be understood as a limitation on the present application.

[0069] The terms "first" and "second" are used for descriptive purposes only and should not be understood as indicating or implying relative importance or implicitly indicating the number of the indicated technical features. Thus, a feature defined as "first" or "second" may explicitly or implicitly include one or more of the features. In the description of this application, unless otherwise specified, "plurality" means two or more.

[0070] In the description of this application, it should be noted that, unless otherwise clearly specified and limited, the terms "installed", "connected", and "connected" should be understood in a broad sense, for example, it can be a fixed connection, a detachable connection, or an integral connection; it can be a mechanical connection or an electrical connection; it can be a direct connection, or it can be indirectly connected through an intermediate medium, or it can be the internal communication of two components. For ordinary technicians in this field, the specific meanings of the above terms in this application can be understood according to specific circumstances.

[0071] like Figure 1 As shown, a dangerous behavior warning method based on driver visual feature fusion in a preferred embodiment of the present application includes:

[0072] S101: generating a multimodal data packet according to the collected multi-source image data, wherein the multimodal data packet includes: primary modal data, secondary modal data and tertiary modal data;

[0073] S102: Processing all data in the multimodal data packet according to a preset fusion model, and generating a fusion feature image according to the fusion result;

[0074] S103: establishing a visual perception model, and generating a driver's visual perception result according to the visual perception model and the fused feature image;

[0075] S104: Generate corresponding warning instructions according to the driver's visual perception results.

[0076] Specifically, when generating a multimodal data packet, it includes:

[0077] Acquire multi-source image data;

[0078] Based on the time series, establish the time sequence T, T = (t1, t2...t m ), where t i is the i-th moment, m is the number of moments;

[0079] Set t in sequence i For the target moment;

[0080] Extract the RGB image features at the target moment in the multi-source image data according to the preprocessing feature model, and generate the primary modality data according to the extraction results;

[0081] Extract infrared features at the target time in multi-source image data according to the preprocessing feature model, and generate secondary modal data according to the extraction results;

[0082] The deep features of the target moment in the multi-source image data are extracted according to the preprocessing feature model, and the tertiary modal data are generated according to the extraction results.

[0083] Specifically, taking ViT as the skeleton network, a pre-processing model is established according to different visual modalities to process the data of the three modalities respectively. A cross-modal feature fusion module is embedded after each layer of ViT to perform feature fusion on the data of each layer.

[0084] Specifically, when generating a fusion feature image according to the fusion result, it includes:

[0085] Establishing a feature correction model;

[0086] Correcting the primary modal data, the secondary modal data, and the tertiary modal data according to the feature correction model;

[0087] Generate first-level features to be fused, second-level features to be fused, and third-level features to be fused according to the correction results;

[0088] According to the deep separable convolution technology, the first-level features to be fused, the second-level features to be fused, and the third-level features to be fused are linearly embedded and fused;

[0089] Generate a fused feature image at the target moment;

[0090] Generate the fusion feature images at each moment in turn, and establish the fusion feature image sequence B, B = (b1, b2...b m ), where b i is the fused feature image at the i-th moment.

[0091] Specifically, due to the complementarity of information between different modalities, the processing of data of different modalities is divided into two steps. The first step is to use the complementarity between different modalities to correct the features of different modalities. The feature correction stage can be divided into the channel-wise stage and the spatial-wise stage. Through the feature correction stage, the features of different modalities can be supplemented and corrected. In the second step, based on the principle of attention mechanism, a cross-modal attention mechanism is designed. First, the corrected feature maps of different modalities are linearly embedded to generate the corresponding Q, K, and V vectors. Then, information is exchanged between different modalities, and feature fusion is performed using deep separable convolution to generate the final fused feature image. The information fusion between different modalities is realized, which improves the accuracy of understanding visual information.

[0092] In a preferred embodiment of the present application, when establishing a visual perception model, it includes:

[0093] Based on the historical parameters, multiple sample training data packets are established, and a sample training data packet sequence W is established, where W = (w1, w2…w n ), where w i is the i-th sample training data packet; n is the number of sample training data packets;

[0094] Select w i Training data packets for target samples;

[0095] Generate a facial image sequence I according to the target sample training data packet, I=(I1,I2…I t );

[0096] The facial image sequence I generates a feature mapping sequence after feature extraction;

[0097] Building a primary processing model based on the feature mapping sequence;

[0098] Generate a loss evaluation value of the primary processing model, and determine whether to generate a correction instruction based on the loss evaluation value;

[0099] Output the gaze prediction sub-model.

[0100] Specifically, the sample training data packet is a collected face video. Based on the Vi T's line of sight estimation model, the timing information between consecutive frames is learned from the video containing the face, so as to enhance the accuracy and generalization performance of the model. The feature data of different modalities are extracted according to the preprocessing feature model, and the fused feature image of the corresponding training sample is generated after correction and fusion.

[0101] Specifically, the loss evaluation value of the model can be set according to the average angular error between the predicted value and the true value. The larger the angular error, the larger the corresponding loss evaluation value.

[0102] Specifically, when determining whether to generate a correction instruction, it includes:

[0103] Preset a first loss evaluation value threshold F1 and a second loss evaluation value threshold F2;

[0104] If f < F1, set the current primary processing model as the line-of-sight prediction sub-model;

[0105] If F1 ≤ f < F2, generate a primary correction instruction, and set the next sample training data packet as the target sample data packet according to the primary correction instruction, and perform iterative optimization on the current primary processing model;

[0106] If f ≥ F2, generate a secondary correction instruction, update the sequence of sample training data packets according to the secondary correction instruction, reselect the target sample data packet according to the update result, and construct a primary processing model.

[0107] Specifically, the primary correction instruction means that the judgment accuracy of the current primary processing model for the line of sight is relatively low, and it is necessary to use the remaining sample training data packets for iterative training to further improve the processing accuracy of the visual processing model.

[0108] Specifically, the secondary correction instruction means that the data feature extraction, calibration, and fusion in the current training sample data packet are relatively poor, resulting in deviation of the primary processing model, and it is necessary to regenerate the corresponding sample training data.

[0109] It can be understood that in the above embodiments, the line-of-sight estimation model based on ViT learns the temporal information between consecutive frames from the video containing the face, thereby enhancing the accuracy and generalization performance of the model.

[0110] Specifically, when establishing the visual perception model, it also includes:

[0111] Generate multiple dangerous behavior data packets based on historical parameters;

[0112] Construct a behavior prediction sub-model according to all the dangerous behavior data packets;

[0113] Construct a visual perception model according to the behavior recognition sub-model and the line-of-sight prediction sub-model.

[0114] Specifically, based on the collected historical parameters, the corresponding behavior prediction sub-model is constructed through iterative training. The behavior prediction sub-model can identify whether the driver in the collected image has dangerous behavior and evaluate the dangerous behavior, thereby generating a behavior risk evaluation value at each moment. The behavior risk evaluation value indicates that the more dangerous behavior exists in the current driving behavior.

[0115] Specifically, dangerous behavior refers to irregular driving behavior. The more dangerous behaviors there are, the greater the possibility of an accident during driving.

[0116] In a preferred embodiment of the present application, when generating the driver's visual perception result, it includes:

[0117] Obtain fused feature image sequence B;

[0118] Generate the sight deviation degree sequence H according to the fused feature image sequence B and the sight prediction sub-model, H = (h1, h2...h m ), where h i is the line of sight deviation at the i-th moment;

[0119] Generate a first reference evaluation value J1 according to the sight deviation degree column H;

[0120]

[0121] Among them, e1 is the preset first weight coefficient, e2 is the preset second weight coefficient, Q1 is the preset first fixed coefficient, Q2 is the preset second fixed coefficient, h' is the safety deviation, Y(i) is the first selection coefficient, if h i -h'≤0, Y(i)=0, if h i -h'>0, Y(i)=1, ΔT is the time interval between adjacent moments.

[0122] Specifically, when generating the driver's visual perception results, it also includes:

[0123] Acquire fused feature image data B;

[0124] Generate a behavior risk assessment value sequence D based on the fused feature image sequence B and the behavior recognition sub-model, D = (d1, d2…d m ), where d i is the behavior risk assessment value at moment i;

[0125] Generate a second reference evaluation value J2 based on the behavior risk evaluation value sequence D

[0126]

[0127] Among them, e3 is a preset third weight coefficient, e4 is a preset fourth weight coefficient, Q3 is a preset third fixed coefficient, Q4 is a preset fourth fixed coefficient, d' is a safety behavior value, P(i) is a second selection coefficient. If d i -d' ≤ 0, P(i) = 0. If d i -d' > 0, P(i) = 1, and ΔT is the time interval between adjacent moments.

[0128] Specifically, corresponding warning instructions are generated according to the driver's visual perception results, including:

[0129] Generate a danger evaluation value k according to the first reference evaluation value J1 and the second reference evaluation value J2;

[0130] k = e5 * J1 + e6 * J2;

[0131] Among them, e5 is a preset fifth weight coefficient, and e6 is a preset sixth weight coefficient;

[0132] Set the danger evaluation value K as the driver's visual perception result, and preset the first danger evaluation value threshold K1 and the second danger evaluation value threshold K2;

[0133] If k ≤ K1, no warning instruction is generated;

[0134] If K1 < k ≤ K2, generate a first-level warning instruction;

[0135] If k > K2, generate a second-level warning instruction.

[0136] Specifically, the first-level warning instruction means that the current driver has problems such as non-standard driving or line-of-sight deviation, and the driver's driving behavior can be corrected through voice reminders. The second-level warning instruction means that when the driver has serious dangerous driving behaviors, warnings need to be given immediately and necessary measures need to be taken to prevent driving accidents. By establishing a danger evaluation model, the warning efficiency for the driver's dangerous behaviors is improved.

[0137] Specifically, the greater the danger evaluation value, the greater the possibility that the current driver has driving risks, and risk warnings need to be given in a timely manner. By establishing a danger evaluation model, the warning efficiency for the driver's dangerous behaviors is improved.

[0138] According to the first concept of the present application, based on the visual multi-modal feature fusion network, corresponding feature preprocessing feature models are established according to different visual modalities to process the feature data of three modalities, correct the different modal feature data by using the complementarity between different modalities, and through the embedded cross-modal feature fusion module, use depthwise separable convolution to fuse the feature data of different modalities to generate a fused feature map. The information fusion between different modalities is realized, and the accuracy of visual information understanding is improved.

[0139] According to the second concept of the present application, by establishing a time series, the image data collected during the driver's driving process is continuously collected, and according to a preset visual processing model, the image data during the driver's driving process is accurately processed to improve the accurate prediction of the driver's line of sight direction and dangerous driving behavior, thereby improving the efficiency of early warning of potential driving risks.

[0140] The above is only a preferred implementation of the present application. It should be pointed out that for ordinary technicians in this technical field, several improvements and substitutions can be made without departing from the technical principles of the present application. These improvements and substitutions should also be regarded as the scope of protection of the present application.

Claims

1. A dangerous behavior warning method based on driver visual feature fusion, characterized in that: Including: Generating a multi-modal data packet based on the collected multi-source image data, where the multi-modal data packet includes: primary modal data, secondary modal data, and tertiary modal data; Processing all the data in the multi-modal data packet according to a preset fusion model, and generating a fusion feature image based on the fusion result; Establishing a visual perception model, and generating a driver's visual perception result based on the visual perception model and the fusion feature image; Generating a corresponding warning instruction according to the driver's visual perception result.

2. The dangerous behavior warning method based on driver visual feature fusion as claimed in claim 1, characterized in that: When generating the multi-modal data packet, it includes: Obtaining multi-source image data; Based on the time series, establish the time sequence T, T = (t1, t2...t m ), where t i is the i-th moment, m is the number of moments; Set t in sequence i For the target moment; Extracting the RGB image features of the target moment in the multi-source image data according to a preprocessing feature model, and generating primary modal data based on the extraction result; Extracting the infrared features of the target moment in the multi-source image data according to a preprocessing feature model, and generating secondary modal data based on the extraction result; Extracting the depth features of the target moment in the multi-source image data according to a preprocessing feature model, and generating tertiary modal data based on the extraction result.

3. The dangerous behavior warning method based on driver visual feature fusion as claimed in claim 2 is characterized in that: When generating a fusion feature image based on the fusion result, it includes: Establishing a feature correction model; Correcting the primary modal data, secondary modal data, and tertiary modal data according to the feature correction model; Generating a primary fusion-ready feature, a secondary fusion-ready feature, and a tertiary fusion-ready feature based on the correction result; Performing linear embedding processing and feature fusion on the primary fusion-ready feature, secondary fusion-ready feature, and tertiary fusion-ready feature according to the depthwise separable convolution technique; Generating a fusion feature image of the target moment; Generate the fusion feature images at each moment in turn, and establish the fusion feature image sequence B, B = (b1, b2...b m ), where b i is the fused feature image at the i-th moment.

4. The dangerous behavior warning method based on driver visual feature fusion as claimed in claim 3 is characterized in that: When establishing a visual perception model, it includes: Based on the historical parameters, multiple sample training data packets are established, and a sample training data packet sequence W is established, where W = (w1, w2…w n ), where w i is the i-th sample training data packet; n is the number of sample training data packets; Select w i Training data packets for target samples; Generate a facial image sequence I according to the target sample training data packet, I=(I1,I2…I t ); Generating a feature mapping sequence after the face image sequence I undergoes feature extraction; Constructing a primary processing model based on the feature mapping sequence; Generating a loss evaluation value of the primary processing model, and determining whether to generate a correction instruction according to the loss evaluation value; Outputting a gaze prediction sub-model.

5. The dangerous behavior warning method based on driver visual feature fusion as claimed in claim 4 is characterized in that: When determining whether to generate a correction instruction, it includes: Presetting a first loss evaluation value threshold F1 and a second loss evaluation value threshold F2; If f < F1, setting the current primary processing model as the gaze prediction sub-model; If F1 ≤ f < F2, generating a primary correction instruction, and setting the next sample training data packet as the target sample data packet according to the primary correction instruction, and performing iterative optimization on the current primary processing model; If f ≥ F2, generating a secondary correction instruction, and updating the sample training data packet sequence according to the secondary correction instruction, re-selecting the target sample data packet according to the update result, and constructing a primary processing model.

6. The dangerous behavior warning method based on driver visual feature fusion as claimed in claim 5, characterized in that: When establishing a visual perception model, it also includes: Generating multiple dangerous behavior data packets based on historical parameters; Constructing a behavior prediction sub-model according to all the dangerous behavior data packets; Constructing a visual perception model according to the behavior recognition sub-model and the gaze prediction sub-model.

7. The dangerous behavior warning method based on driver visual feature fusion as claimed in claim 6, characterized in that: When generating a driver's visual perception result, it includes: Obtaining a fusion feature image sequence B; Generate the sight deviation degree sequence H according to the fused feature image sequence B and the sight prediction sub-model, H = (h1, h2...h m ), where h i is the line of sight deviation at the i-th moment; Generating a first reference evaluation value J1 according to the gaze deviation sequence H; Among them, e1 is the preset first weight coefficient, e2 is the preset second weight coefficient, Q1 is the preset first fixed coefficient, Q2 is the preset second fixed coefficient, h' is the safety deviation, Y(i) is the first selection coefficient, if h i -h'≤0, Y(i)=0, if h i -h'>0, Y(i)=1, ΔT is the time interval between adjacent moments.

8. The dangerous behavior warning method based on driver visual feature fusion as claimed in claim 7 is characterized in that: When generating a driver's visual perception result, it also includes: Obtaining fusion feature image data B; Generate a behavior risk assessment value sequence D based on the fused feature image sequence B and the behavior recognition sub-model, D = (d1, d2…d m ), where d i is the behavior risk assessment value at moment i; Generating a second reference evaluation value J2 according to the behavior risk evaluation value sequence D Among them, e3 is the preset third weight coefficient, e4 is the preset fourth weight coefficient, Q3 is the preset third fixed coefficient, Q4 is the preset fourth fixed coefficient, d' is the safety behavior value, P(i) is the second selection coefficient, if d i -d'≤0, P(i)=0, if d i -d'>0, P(i)=1, ΔT is the time interval between adjacent moments.

9. The dangerous behavior warning method based on driver visual feature fusion as claimed in claim 8, characterized in that: When generating a corresponding warning instruction according to the driver's visual perception result, it includes: Generating a danger evaluation value k according to the first reference evaluation value J1 and the second reference evaluation value J2; k = e5 * J1 + e6 * J2; Wherein, e5 is a preset fifth weight coefficient, and e6 is a preset sixth weight coefficient; Set the danger evaluation value K as the driver's visual perception result, and preset a first danger evaluation value threshold K1 and a second danger evaluation value threshold K2; If k ≤ K1, no warning command is generated; If K1 < k ≤ K2, a first-level warning command is generated; If k > K2, a second-level warning command is generated.

Images