Fatigue state identification method and device

Abstract

The embodiment of the invention provides a fatigue state identification method and device, and relates to the technical field of fatigue driving detection. The electronic equipment obtains driving features and current scene information of a driver to be recognized, and then inputs the driving features into the fatigue feature extraction model for convolution processing to obtain driver fatigue features output by the fatigue feature extraction model; determining a preset number of pieces of historical scene information matched with the current scene information from historical scene information of a pre-established scene feature library according to a scene matching algorithm as target scene information; determining target fatigue features corresponding to the target scene information according to the corresponding relation of the scene feature library, and determining the fatigue similarity between the driver fatigue feature and each target fatigue feature, thereby determining the fatigue state of the to-be-recognized driver based on the fatigue similarity, and determining the fatigue degree of the to-be-recognized driver from multiple dimensions. And the accuracy of identifying the fatigue state is improved.

Description

Technical Field

[0001] The present application relates to the technical field of fatigue driving detection, and in particular to a method and device for identifying a fatigue state. Background Art

[0002] With the rapid development of the vehicle manufacturing industry, the frequency of traffic accidents is also gradually increasing, among which driver fatigue driving accounts for a high proportion of the causes of traffic accidents. When the driver is driving a vehicle in a fatigued state, the driver's attention, thinking, judgment and other aspects are limited, resulting in a decline in driving skills, which is easy to cause traffic accidents.

[0003] Therefore, fatigue driving detection technology plays a vital role in traffic safety. In the existing fatigue driving recognition method, whether the driver closes his eyes is generally judged based on the degree of eye opening, but this method cannot solve the problem of false alarm and missed alarm of fatigue driving in scenes where the driver looks down, squints, has small eyes, and the driver's image is blurred. Therefore, the accuracy of fatigue state recognition is not high. Summary of the invention

[0004] The purpose of the embodiments of the present application is to provide a method and device for identifying a fatigue state, so as to solve the problem of low accuracy in identifying a fatigue state. The specific technical solution is as follows:

[0005] In a first aspect, an embodiment of the present application provides a method for identifying a fatigue state, the method comprising:

[0006] Acquiring driving characteristics of the driver to be identified and current scene information, wherein the driving characteristics represent the state of the driver to be identified and the driving state of the vehicle;

[0007] Inputting the driving feature into a pre-trained fatigue feature extraction model for convolution processing to obtain the driver fatigue feature output by the fatigue feature extraction model, wherein the fatigue feature extraction model includes a plurality of convolution layers;

[0008] Determining, from historical scene information in a pre-established scene feature library, a preset number of historical scene information matching the current scene information according to a scene matching algorithm as target scene information, wherein the scene feature library includes a correspondence between historical scene information and historical fatigue features;

[0009] According to the corresponding relationship, determining the target fatigue feature corresponding to the target scene information, and determining the fatigue similarity between the driver fatigue feature and each target fatigue feature;

[0010] Based on the fatigue similarity, the fatigue state of the driver to be identified is determined.

[0011] Optionally, the training method of the fatigue feature extraction model includes:

[0012] Obtain driving feature samples and fatigue confidence truth values ​​corresponding to each driving feature sample;

[0013] Inputting the driving feature sample into an initial fatigue feature extraction model so that a plurality of convolutional layers included in the initial fatigue feature extraction model perform convolution processing on the driving feature sample and outputting the obtained predicted fatigue feature to a fully connected network;

[0014] The fully connected network processes the predicted fatigue feature to obtain a one-dimensional fatigue feature vector corresponding to the predicted fatigue feature;

[0015] Mapping the one-dimensional fatigue feature vector using an activation function to obtain an estimated fatigue confidence corresponding to the one-dimensional fatigue feature vector;

[0016] Calculating the loss value between the estimated fatigue confidence and the corresponding fatigue confidence true value using a loss function;

[0017] The parameters of the convolutional layer are adjusted based on the loss value until the initial fatigue feature extraction model converges to obtain the fatigue feature extraction model.

[0018] Optionally, before the step of determining, from the historical scene information in the pre-established scene feature library, a preset number of historical scene information matching the current scene information according to a scene matching algorithm, the method further comprises:

[0019] Acquire historical scene information and a driving characteristic of a driver acquired in a scene corresponding to the historical scene information as a historical driving characteristic;

[0020] Inputting the historical driving features into a pre-trained fatigue feature extraction model for convolution processing to obtain historical fatigue features output by the fatigue feature extraction model;

[0021] A corresponding relationship between the historical scene information and the historical fatigue features is established and stored to obtain a scene feature library.

[0022] Optionally, the step of determining, from the historical scene information in the pre-established scene feature library, a preset number of historical scene information matching the current scene information according to a scene matching algorithm as the target scene information includes:

[0023] For each historical scene information included in the scene feature library, determining the scene similarity between the current scene information and the historical scene information according to a scene matching algorithm;

[0024] A preset number of historical scene information with the highest corresponding scene similarities are selected from the historical scene information as target scene information.

[0025] Optionally, the current scene information and the historical scene information both include scene information of at least one dimension;

[0026] The step of determining the scene similarity between the current scene information and the historical scene information according to the scene matching algorithm comprises:

[0027] Determine a degree of difference between the scene information of each dimension included in the current scene information and the scene information of the dimension included in the historical scene information;

[0028] The scene similarity between the current scene information and the historical scene information is determined according to the degree of difference corresponding to the scene information of each dimension.

[0029] Optionally, the at least one dimension of scene information includes at least one of driving time period, weather, road type, continuous driving time, and average eye distance;

[0030] The step of determining the degree of difference between the scene information of each dimension included in the current scene information and the scene information of the dimension included in the historical scene information comprises:

[0031] In the case where the scenario information includes a driving period, determining a time difference degree between the current driving period and the historical driving period according to a minimum time difference between a current driving period included in the current scenario information and a historical driving period included in the historical scenario information;

[0032] In the case where the scene information includes weather, determining the degree of weather difference between the current weather and the historical weather according to whether the current weather included in the current scene information is consistent with the historical weather included in the historical scene information;

[0033] In the case where the scene information includes a road type, determining a degree of difference in road type between the current road type and the historical road type according to whether the current road type included in the current scene information is consistent with the historical road type included in the historical scene information;

[0034] In the case where the scenario information includes continuous driving duration, determining the duration difference between the current continuous driving duration and the historical continuous driving duration according to the minimum duration difference between the current continuous driving duration included in the current scenario information and the historical continuous driving duration included in the historical scenario information;

[0035] In the case where the scene information includes the average eye distance, the degree of difference in the eye distance between the current average eye distance and the historical average eye distance is determined according to the minimum average eye distance difference between the current average eye distance included in the current scene information and the historical average eye distance included in the historical scene information.

[0036] Optionally, the step of determining the degree of time difference between the current driving period and the historical driving period according to the minimum time difference between the current driving period included in the current scenario information and the historical driving period included in the historical scenario information comprises:

[0037] The difference between the current driving period and the historical driving period is calculated according to the following formula: :

[0038] ;

[0039] Among them, the is the current driving period, For historical driving time;

[0040] The step of determining the degree of weather difference between the current weather and the historical weather according to whether the current weather included in the current scene information is consistent with the historical weather included in the historical scene information comprises:

[0041] The degree of weather difference between the current weather and the historical weather is calculated according to the following formula :

[0042] ;

[0043] Among them, the For the current weather, For historical weather;

[0044] The step of determining the degree of difference between the current road type and the historical road type according to whether the current road type included in the current scene information is consistent with the historical road type included in the historical scene information comprises:

[0045] The degree of difference between the current road type and the historical road type is calculated according to the following formula :

[0046] ;

[0047] Among them, the is the current road type, It is a historical road type;

[0048] The step of determining the degree of difference between the current continuous driving duration and the historical continuous driving duration according to the minimum duration difference between the current continuous driving duration included in the current scene information and the historical continuous driving duration included in the historical scene information comprises:

[0049] The difference between the current continuous driving duration and the historical continuous driving duration is calculated according to the following formula: :

[0050] ;

[0051] Among them, the is the current continuous driving time, The historical continuous driving time;

[0052] The step of determining the degree of difference between the current average eye distance and the historical average eye distance according to the minimum average eye distance difference between the current average eye distance included in the current scene information and the historical average eye distance included in the historical scene information comprises:

[0053] The difference between the current average eye distance and the historical average eye distance is calculated according to the following formula: :

[0054] ;

[0055] Among them, the is the current average eye distance, The historical average eye distance.

[0056] Optionally, the step of determining the scene similarity between the current scene information and the historical scene information according to the degree of difference corresponding to the scene information of each dimension includes:

[0057] The scene similarity between the current scene information and the historical scene information is calculated according to the following formula :

[0058] ;

[0059] Among them, the is the current scene information, is the historical scene information, is the degree of difference between the time periods, is the degree of weather difference, is the degree of difference of the road types, is the duration difference, is the degree of difference in interocular distance.

[0060] Optionally, the step of determining the fatigue similarity between the driver fatigue feature and each target fatigue feature includes:

[0061] Calculating the cosine similarity between the driver fatigue feature and each target fatigue feature;

[0062] The average value of the cosine similarities is calculated as the fatigue similarity.

[0063] Optionally, the step of determining the fatigue state of the driver to be identified based on the fatigue similarity includes:

[0064] Based on the difference between the fatigue similarity and a preset value, it is determined that the fatigue state of the driver to be identified is fatigue driving or normal driving, wherein the preset value is used to characterize the fatigue similarity corresponding to fatigue driving or normal driving.

[0065] Optionally, the step of acquiring the driving characteristics of the driver to be identified includes:

[0066] When the driver to be identified is driving the vehicle, an in-vehicle image captured by an in-vehicle image sensor within a preset time period is obtained, and image recognition is performed on the in-vehicle image to obtain at least one of the eye opening degree, mouth opening degree, sight direction angle, head posture angle, and human body key points of the driver to be identified as an in-vehicle parameter;

[0067] Acquire an image outside the vehicle collected by an image sensor outside the vehicle within the preset time period, and perform image recognition on the image outside the vehicle to obtain at least one of a lane deviation distance, a number of surrounding vehicles, and a front vehicle collision distance as an external vehicle parameter;

[0068] Acquire at least one of the vehicle speed, acceleration and steering wheel angle of the vehicle within the preset time period as a driving parameter;

[0069] The in-vehicle parameters, the out-vehicle parameters and the driving parameters are combined into a feature vector of a preset dimension according to the acquisition time to obtain the driving characteristics of the driver to be identified.

[0070] In a second aspect, an embodiment of the present application provides a fatigue state recognition device, the device comprising:

[0071] A feature information acquisition module, used to acquire driving features of the driver to be identified and current scene information, wherein the driving features represent the state of the driver to be identified and the driving state of the vehicle;

[0072] A fatigue feature acquisition module, used for inputting the driving feature into a pre-trained fatigue feature extraction model for convolution processing to obtain the driver fatigue feature output by the fatigue feature extraction model, wherein the fatigue feature extraction model includes a plurality of convolution layers;

[0073] A target scene information determination module, used to determine a preset number of historical scene information matching the current scene information from historical scene information in a pre-established scene feature library according to a scene matching algorithm as target scene information, wherein the scene feature library includes a correspondence between historical scene information and historical fatigue features;

[0074] a fatigue similarity determination module, configured to determine, based on the corresponding relationship, a target fatigue feature corresponding to the target scene information, and determine a fatigue similarity between the driver fatigue feature and each target fatigue feature;

[0075] A fatigue state determination module is used to determine the fatigue state of the driver to be identified based on the fatigue similarity.

[0076] In a third aspect, an embodiment of the present application provides an electronic device, including:

[0077] Memory, used to store computer programs;

[0078] The processor is used to implement any method described in the first aspect when executing a program stored in the memory.

[0079] In a fourth aspect, an embodiment of the present application provides a computer-readable storage medium, wherein the computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, any method described in the first aspect is implemented.

[0080] Beneficial effects of the embodiments of the present application:

[0081] In the solution provided by the embodiment of the present application, the electronic device can obtain the driving characteristics and current scene information of the driver to be identified, wherein the driving characteristics represent the state of the driver to be identified and the driving state of the vehicle, and then input the driving characteristics into the pre-trained fatigue feature extraction model for convolution processing to obtain the driver fatigue characteristics output by the fatigue feature extraction model, wherein the fatigue feature extraction model includes multiple convolution layers; then from the historical scene information of the pre-established scene feature library, according to the scene matching algorithm, a preset number of historical scene information matching the current scene information is determined as the target scene information, wherein the scene feature library includes the corresponding relationship between the historical scene information and the historical fatigue characteristics; then according to the corresponding relationship, the target fatigue characteristics corresponding to the target scene information are determined, and the fatigue similarity between the driver fatigue characteristics and each target fatigue characteristic is determined, so as to determine the fatigue state of the driver to be identified based on the fatigue similarity. Since the fatigue states corresponding to the same scene have similarities, the current scene information is matched with the historical scene information of the pre-established scene feature library, so that the determined target fatigue characteristics are capable of representing the characteristics of the fatigue state under the current scene information. Then, based on the fatigue similarity between the driver's fatigue feature and each target fatigue feature, the fatigue state of the driver to be identified is obtained, and the driver's fatigue state can be accurately determined. The fatigue degree of the driver to be identified is determined through information from multiple dimensions, thereby improving the accuracy of identifying the fatigue state. Of course, implementing any product or method of the present application does not necessarily require achieving all of the advantages described above at the same time. BRIEF DESCRIPTION OF THE DRAWINGS

[0082] In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application, and for ordinary technicians in this field, other embodiments can also be obtained based on these drawings.

[0083] Figure 1 A flowchart of a fatigue state recognition method provided in an embodiment of the present application;

[0084] Figure 2(a) is based on Figure 1 A schematic diagram of the driver's gaze downward while driving a vehicle in the illustrated embodiment;

[0085] Figure 2(b) is based on Figure 1 A schematic diagram of a driver squinting while driving a vehicle in the illustrated embodiment;

[0086] Figure 2(c) is based on Figure 1 The driver of the illustrated embodiment is a schematic diagram of a small eye;

[0087] Figure 2(d) is based on Figure 1 A schematic diagram of a driver image blurring in the illustrated embodiment;

[0088] Figure 3 Based on Figure 1 A flow chart of training a fatigue feature extraction model according to the embodiment shown;

[0089] Figure 4 for Figure 1 A specific flow chart of step S103 in the illustrated embodiment;

[0090] Figure 5 Based on Figure 1 A schematic diagram of a fatigue state identification method of the illustrated embodiment;

[0091] Figure 6 A schematic diagram of the structure of a fatigue state recognition device provided in an embodiment of the present application;

[0092] Figure 7 A schematic diagram of the structure of an electronic device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0093] The following will be combined with the drawings in the embodiments of the present application to clearly and completely describe the technical solutions in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field based on the present application belong to the scope of protection of the present application.

[0094] In order to avoid the problem of false alarm and missed alarm of fatigue driving in the process of fatigue driving identification and improve the accuracy of fatigue state identification, the present application embodiment provides a fatigue state identification method and device. The following first introduces a fatigue state identification method provided by the present application embodiment.

[0095] A fatigue status identification method provided in an embodiment of the present application can be applied to any electronic device that needs to identify the driver's fatigue status, for example, it can be a vehicle-mounted host, a fatigue driving warning device, a DMS (Driver Monitor System), etc., without specific limitation here. For the sake of clear description, it is referred to as an electronic device in the following.

[0096] like Figure 1 As shown, a method for identifying a fatigue state comprises:

[0097] S101, obtaining driving characteristics and current scene information of the driver to be identified;

[0098] The driving characteristics represent the state of the driver to be identified and the driving state of the vehicle.

[0099] S102, inputting the driving feature into a pre-trained fatigue feature extraction model for convolution processing to obtain a driver fatigue feature output by the fatigue feature extraction model;

[0100] Wherein, the fatigue feature extraction model includes multiple convolutional layers.

[0101] S103, determining a preset number of historical scene information matching the current scene information from the historical scene information in a pre-established scene feature library according to a scene matching algorithm as target scene information;

[0102] The scene feature library includes the correspondence between historical scene information and historical fatigue features.

[0103] S104, determining the target fatigue feature corresponding to the target scene information according to the corresponding relationship, and determining the fatigue similarity between the driver fatigue feature and each target fatigue feature.

[0104] S105: Determine the fatigue state of the driver to be identified based on the fatigue similarity.

[0105] It can be seen that in the solution provided by the embodiment of the present application, the electronic device can obtain the driving characteristics and current scene information of the driver to be identified, wherein the driving characteristics represent the state of the driver to be identified and the driving state of the vehicle, and then input the driving characteristics into the pre-trained fatigue feature extraction model for convolution processing to obtain the driver fatigue characteristics output by the fatigue feature extraction model, wherein the fatigue feature extraction model includes multiple convolution layers; then from the historical scene information of the pre-established scene feature library, according to the scene matching algorithm, a preset number of historical scene information matching the current scene information is determined as the target scene information, wherein the scene feature library includes the corresponding relationship between the historical scene information and the historical fatigue characteristics; then according to the corresponding relationship, the target fatigue characteristics corresponding to the target scene information are determined, and the fatigue similarity between the driver fatigue characteristics and each target fatigue characteristic is determined, so as to determine the fatigue state of the driver to be identified based on the fatigue similarity. Since the fatigue states corresponding to the same scene have similarities, the current scene information is matched with the historical scene information of the pre-established scene feature library, so that the determined target fatigue characteristics are capable of representing the characteristics of the fatigue state under the current scene information. Then, based on the fatigue similarity between the driver's fatigue characteristics and each target fatigue characteristic, the fatigue state of the driver to be identified can be obtained, and the driver's fatigue state can be accurately determined. The fatigue degree of the driver to be identified is determined through information from multiple dimensions, thereby improving the accuracy of identifying the fatigue state.

[0106] When a driver is driving a vehicle in a fatigued state, the driver will feel sleepy, blurry-eyed, weak limbs, difficulty concentrating, and even in a trance. Usually, when a driver is driving a vehicle, the order of physical fatigue is the eyes, neck, shoulders, and waist. Therefore, in the existing fatigue driving detection technology, it is usually based on a single eye signal of the driver's eye opening degree to determine whether the driver has closed his eyes. When the driver has closed his eyes, it can be determined that the driver is in a fatigued state.

[0107] However, according to the existing fatigue driving detection technology, if the driver looks downward while driving the vehicle as shown in Figure 2 (a), squints due to strong light as shown in Figure 2 (b), or has small eyes as shown in Figure 2 (c), the driver may be mistakenly detected as being in a fatigued state. For the blurred image of the driver taken by the image acquisition device as shown in Figure 2 (d), it is difficult to accurately detect whether the driver is in a fatigued state based on this blurred image.

[0108] In order to more accurately identify whether the driver is in a fatigue state, an embodiment of the present application provides a method for identifying a fatigue state. In step S101, the electronic device can obtain the driving characteristics and current scene information of the driver to be identified. Among them, the driving characteristics can characterize the state of the driver to be identified and the driving state of the vehicle. The electronic device processes the recognition signal of the image sensor installed in the vehicle to obtain the state of the driver to be identified, and processes the recognition signal of the image sensor installed outside the vehicle and the CAN (Controller Area Network) signal to obtain the driving state of the vehicle.

[0109] Among them, the electronic device can obtain at least one parameter of the driver's eye opening degree, mouth opening degree, sight direction angle, head posture angle, or key points of the human body through the recognition signal of the in-vehicle image sensor, and the features composed of these parameters can be used to characterize the state of the driver to be identified. The electronic device can obtain at least one parameter of lane offset distance, number of surrounding vehicles, or front vehicle collision distance through the recognition signal of the external image sensor, and can obtain at least one parameter of vehicle speed, acceleration, and steering wheel angle through the CAN signal, and the features composed of these parameters can be used to characterize the driving state of the vehicle.

[0110] The current scene information can characterize the environment in which the driver is currently driving the vehicle. For example, it can be the road conditions and weather conditions of the vehicle currently driving obtained through the satellite positioning system, the time period and driving duration of the driver driving the vehicle obtained through the driving recorder, and the average eye distance of the driver obtained by analyzing the driver image taken by the image acquisition device. These are all reasonable and are not specifically limited here.

[0111] After obtaining the driving characteristics of the driver to be identified and the current scene information, in the above step S102, the electronic device can input the driving characteristics into a pre-trained fatigue feature extraction model for convolution processing to obtain the driver fatigue characteristics output by the fatigue feature extraction model. The fatigue feature extraction model may include multiple convolution layers, and each convolution layer includes multiple convolution kernels.

[0112] The electronic device can input the driving characteristics into a pre-trained fatigue feature extraction model, and the fatigue feature extraction model performs convolution calculation processing on the driving characteristics, slides the convolution kernel on the fatigue feature vector corresponding to the fatigue characteristics, extracts the local characteristics of the fatigue characteristics through convolution operation, and obtains the driver fatigue characteristics after multiple convolution processing. Specifically, after multiple convolution calculations, the fatigue feature extraction model will generate a series of hidden feature vectors as the driver fatigue characteristics.

[0113] The driver fatigue feature vector output by the fatigue feature extraction model is more abstract than the driving feature, contains a higher level of feature representation, and can more accurately characterize the state of the driver to be identified and the intrinsic properties of the vehicle's driving state. Compared with the driving feature, the driver fatigue feature obtained after multiple convolution operations can more accurately capture the recognition signals of the in-vehicle image sensor, the recognition signals of the out-vehicle image sensor, and the spatial and temporal dependencies in the CAN signal.

[0114] After acquiring the current scene information, in the above step S103, the electronic device can determine a preset number of historical scene information matching the current scene information from the historical scene information in the pre-established scene feature library according to the scene matching algorithm as the target scene information.

[0115] The above-mentioned scene information may include at least one of driving time period, weather, road type, continuous driving time and average eye distance, which is not specifically limited here.

[0116] Due to different scene information, the electronic device may be affected to a certain extent when determining the fatigue state of the person to be identified. For example, when the scene information includes the weather, if the weather in the current scene information is sunny, the driver is easily affected by the sun's rays and may squint while driving the vehicle; at this time, if the historical scene information of cloudy weather is determined as the target scene information, and the target fatigue state corresponding to the target scene information is fatigue driving, the fatigue state of the driver to be identified may be determined as fatigue driving, resulting in misjudgment.

[0117] The fatigue states corresponding to the same scene are similar and have a higher reference value. Therefore, the electronic device can determine a preset number of historical scene information as target scene information from the historical scene information in the scene feature library according to the scene matching algorithm. The preset number of historical scene information has a high similarity with the current scene information, which makes it convenient for the electronic device to further determine the target fatigue feature after determining the target scene information based on the correspondence between the historical scene information and the historical fatigue features included in the scene feature library.

[0118] In one embodiment, the correspondence between the historical scene information and the historical fatigue features included in the scene feature library can be stored in the form of key-value pairs, or can be stored through a database table, or can be stored in the form of a mapping table, which is not specifically limited here.

[0119] Among them, the above preset number can be set according to actual conditions. In the case where the historical scene information in the scene feature library includes scene information of multiple dimensions, in order to reduce the amount of calculation, the preset number can be set to be relatively small. For example, when the historical scene information includes scene information of 4 dimensions, the preset number can be 15; when the historical scene information includes scene information of 5 dimensions, the preset number can be 10. In the case where the historical scene information in the scene feature library only includes scene information of one dimension, in order to ensure the accuracy of scene matching, the preset number can be 20, which is reasonable and is not specifically limited here.

[0120] In the above step S104, after determining a preset number of target scene information matching the current scene information in the scene feature library, the electronic device can determine the target fatigue features corresponding to the above target scene information based on the correspondence between the historical scene information and the historical fatigue features, and determine the fatigue similarity between the driver's fatigue features and each target fatigue feature.

[0121] In one embodiment, the electronic device can determine the fatigue similarity by calculating the cosine similarity between the driver's fatigue feature and each target fatigue feature, or by calculating the Euclidean distance between the two, or by calculating the Manhattan distance between the two, which are all reasonable and are not specifically limited here.

[0122] After determining the fatigue similarity between the driver's fatigue feature and each target fatigue feature, in the above step S105 , the electronic device determines the fatigue state of the driver to be identified based on the fatigue similarity.

[0123] Since a preset number of target scene information is determined in the historical scene information in the scene feature library, and a preset number of target fatigue features corresponding to the target scene information can be determined according to the correspondence between the historical scene information and the historical fatigue features, the electronic device can determine a preset number of fatigue similarities corresponding to the target fatigue features one by one by using the similarity calculation method, and save these fatigue similarities one by one.

[0124] In one embodiment, when determining the fatigue state of the driver to be identified based on a preset number of fatigue similarities, the average value of the preset number of fatigue similarities can be calculated, and then the fatigue state of the person to be identified can be determined by the average value of the fatigue similarities. In addition, the fatigue state of the driver to be identified can also be determined based on the maximum value or median value of the preset number of fatigue similarities, which is not specifically limited here.

[0125] Since the target fatigue feature is the fatigue feature corresponding to the fatigue driving state under the target scene information, and the current scene information is similar to the target scene information, the higher the similarity between the fatigue feature of the driver to be identified and the target fatigue feature, the greater the possibility that the driver to be identified is currently in a fatigue driving state.

[0126] As an implementation method, a similarity threshold can be set in advance, such as 0.8, 0.85, 0.9, etc. When the fatigue similarity is not less than the similarity threshold, it means that the similarity between the fatigue characteristics of the driver to be identified and the target fatigue characteristics is very high, then it can be determined that the fatigue state of the driver to be identified is fatigue driving; when the fatigue similarity is less than the similarity threshold, it means that the similarity between the fatigue characteristics of the driver to be identified and the target fatigue characteristics is low, then it can be determined that the fatigue state of the driver to be identified is normal driving.

[0127] It can be seen that in this embodiment, since the fatigue states corresponding to the same scene have similarities, the current scene information is matched with the historical scene information of the pre-established scene feature library, so that the target fatigue feature determined is capable of characterizing the characteristics of the fatigue state under the current scene information. Then, based on the fatigue similarity between the driver's fatigue feature and each target fatigue feature, the fatigue state of the driver to be identified is obtained, and the driver's fatigue state can be accurately determined. The fatigue degree of the driver to be identified is determined through information from multiple dimensions, which improves the accuracy of identifying the fatigue state.

[0128] As an implementation method of the present application, Figure 3 As shown, the training method of the fatigue feature extraction model may include:

[0129] S301, obtaining driving feature samples and a fatigue confidence truth value corresponding to each driving feature sample.

[0130] The driving feature samples may include fatigue driving feature samples and normal driving feature samples, which are respectively used as positive samples and negative samples. The fatigue driving feature samples may be divided into multiple levels according to the degree of fatigue. When training the initial fatigue feature extraction model, the higher the level of the fatigue driving feature sample, the closer the corresponding fatigue confidence true value is to 1.

[0131] The electronic device can obtain multiple driving feature samples and the fatigue confidence true value corresponding to each driving feature sample, wherein the above-mentioned driving feature samples can be obtained by processing the historical recognition signals of image sensors inside and outside the vehicle, and the corresponding historical CAN signals; the fatigue confidence true value corresponding to each driving feature sample represents the fatigue state of the driver corresponding to the driving feature sample.

[0132] S302, inputting the driving feature sample into the initial fatigue feature extraction model, so that the multiple convolution layers included in the initial fatigue feature extraction model perform convolution processing on the driving feature sample, and outputting the obtained predicted fatigue feature to the fully connected network.

[0133] After obtaining the driving feature samples, the electronic device can input the driving feature samples into an initial fatigue feature extraction model, wherein the initial fatigue feature extraction model can be a neural network model including multiple convolutional layers, using multiple convolution kernels corresponding to each convolutional layer to slide on the driving feature sample vector of the driving feature sample, extracting the local features of the driving feature sample as predicted fatigue features through convolution operation processing, and outputting the predicted fatigue features to a fully connected network.

[0134] S303, the fully connected network processes the predicted fatigue feature to obtain a one-dimensional fatigue feature vector corresponding to the predicted fatigue feature.

[0135] The fully connected network flattens the obtained predicted fatigue features into a one-dimensional fatigue feature vector, and connects the one-dimensional fatigue feature vector to the output node through the fully connected weight matrix to classify it. The one-dimensional fatigue feature vector obtained by the fully connected network has a smaller dimension than the above-mentioned predicted fatigue features, which is convenient for subsequent further calculation and processing.

[0136] S304, mapping the one-dimensional fatigue feature vector using an activation function to obtain an estimated fatigue confidence level corresponding to the one-dimensional fatigue feature vector.

[0137] After obtaining the above one-dimensional fatigue feature vector, the estimated fatigue confidence obtained by mapping the one-dimensional fatigue feature vector using the activation function is in the range of 0 to 1, thereby making the output of the initial fatigue feature extraction model more stable, which helps to improve the convergence speed and performance of the model.

[0138] The activation function may be a Sigmoid operator, which is used to classify the one-dimensional feature vector into two categories, thereby mapping it to an estimated fatigue confidence interval of (0, 1). The activation function may also be a Tanh activation function or a ReLU activation function, which is not specifically limited here.

[0139] S305: Calculate the loss value between the estimated fatigue confidence and the corresponding fatigue confidence true value using a loss function.

[0140] After obtaining the estimated fatigue confidence, the electronic device can use the loss function to calculate the loss value between the estimated fatigue confidence and the corresponding fatigue confidence true value, and the loss value represents the difference between the estimated fatigue confidence output by the current initial fatigue feature extraction model and the driving feature sample and the corresponding fatigue confidence true value. The above loss function can be a mean square error loss function or a cross entropy loss function, which is not specifically limited here.

[0141] The smaller the loss value calculated by the loss function, the more accurate the predicted fatigue features output by the current initial fatigue feature extraction model are. The larger the loss value calculated by the loss function, the less accurate the predicted fatigue features output by the current initial fatigue feature extraction model are.

[0142] S306, adjusting the parameters of the convolution layer based on the loss value until the initial fatigue feature extraction model converges to obtain a fatigue feature extraction model.

[0143] After the loss value is calculated, the electronic device can adjust the model parameters of multiple convolutional layers included in the current initial fatigue feature extraction model to change the loss value in the direction of the fastest reduction.

[0144] During the training process, the model parameters of the initial fatigue feature extraction model can be continuously adjusted to make the difference between the estimated fatigue confidence and the corresponding fatigue confidence true value gradually smaller. The initial fatigue feature extraction model can continuously learn the correspondence between driving characteristics and fatigue confidence until the loss value between the estimated fatigue confidence and the corresponding fatigue confidence true value basically no longer changes. Alternatively, when the number of iterations reaches a preset number, the initial fatigue feature extraction model converges. At this time, the initial fatigue feature extraction model can output accurate driver fatigue features, then the training can be stopped, and the current initial fatigue feature extraction model is the fatigue feature extraction model.

[0145] The method for adjusting the parameters of the initial fatigue feature extraction model may specifically adopt a gradient descent algorithm, a stochastic gradient descent algorithm, etc., which is not specifically limited here.

[0146] It can be seen that in this embodiment, the electronic device can construct a loss function based on the difference between the estimated fatigue confidence and the corresponding fatigue confidence true value, and then train the initial fatigue feature extraction model based on multiple driving feature samples and loss values ​​until the above loss value reaches convergence to obtain a fatigue feature extraction model. In this way, the fatigue feature extraction model is obtained by training the initial fatigue feature extraction model using driving feature samples, and accurate driver fatigue features can be extracted.

[0147] As an implementation of an embodiment of the present application, before the step of determining a preset number of historical scene information matching the current scene information from the historical scene information in the pre-established scene feature library according to a scene matching algorithm, the method may further include:

[0148] The historical scene information and the driving characteristics of the driver obtained in the scene corresponding to the historical scene information are obtained as the historical driving characteristics; the historical driving characteristics are input into a pre-trained fatigue feature extraction model for convolution processing to obtain the historical fatigue characteristics output by the fatigue feature extraction model; the corresponding relationship between the historical scene information and the historical fatigue characteristics is established and stored to obtain a scene feature library.

[0149] When establishing a scene feature library, the electronic device can obtain historical scene information and the driver's driving characteristics obtained in the scene corresponding to the historical scene information as historical driving characteristics, wherein the historical driving characteristics include the driving characteristics of the driver in a fatigue driving state.

[0150] Then, the historical fatigue driving features are input into the pre-trained fatigue feature extraction model. After the convolution operation of the historical fatigue driving features by multiple convolution layers in the fatigue feature extraction model, the historical fatigue features corresponding to the historical driving features output by the fatigue feature extraction model can be obtained. Based on this, the corresponding relationship between the historical scene information and the historical fatigue features can be stored to obtain a scene feature library.

[0151] Among them, in the established scene feature library, the correspondence between historical scene information and historical fatigue features can be stored in the form of key-value pairs, or stored through a two-dimensional database table, or stored in the form of a tree diagram, which is not specifically limited here.

[0152] For example, taking historical scene information including driving time, weather, road type, continuous driving time and average eye distance as an example, the scene feature library stored in the database table can be as follows:

[0153]

[0154] It can be seen that in this embodiment, the electronic device converts the driver's fatigue state recognition problem into a feature similarity matching problem between the driver's fatigue features and the target fatigue features by establishing a scene feature library including the correspondence between historical scene information and historical fatigue features, and selecting target fatigue features from the historical fatigue features. This can better utilize historical experience and rules, improve the accuracy of fatigue judgment, and avoid the conventional method of fatigue model often using classification or confidence regression to determine fatigue state. In addition, by using the scene information library for feature matching, as the historical scene information and historical fatigue features in the scene feature library gradually accumulate, the final effect of fatigue state recognition will gradually improve. The more historical fatigue features there are and the richer the historical scene information is, the better the effect of fatigue state recognition will be.

[0155] As an implementation method of an embodiment of the present application, the step of determining a preset number of historical scene information matching the current scene information from the historical scene information in a pre-established scene feature library according to a scene matching algorithm as the target scene information may include:

[0156] For each historical scene information included in the above scene feature library, the scene similarity between the current scene information and the historical scene information is determined according to the scene matching algorithm; from the historical scene information, a preset number of historical scene information with the highest corresponding scene similarity are selected as the target scene information.

[0157] Since confirming fatigue similarity under similar scene information can improve the accuracy of fatigue state recognition, after acquiring the current scene information, the electronic device determines the scene similarity between the current scene information and each historical scene information in the scene feature library according to the scene matching algorithm, and then selects the TopN historical scene information with the highest similarity as the target scene information according to the level of scene similarity, and marks these N target scene information as Top1 to TopN according to the level of scene similarity.

[0158] Then, based on the correspondence between the historical scene information and the historical fatigue status included in the scene feature library, N target fatigue features corresponding to the target scene information from Top1 to TopN can be determined for subsequent determination of the fatigue similarity between the driver fatigue feature and each of the N target fatigue features.

[0159] Among them, the above-mentioned preset number can be set according to factors such as the amount of historical scene information, the data processing capability of the electronic device, and the requirements for fatigue similarity accuracy, which are all reasonable and are not specifically limited here.

[0160] It can be seen that in this embodiment, the electronic device can select a preset number of historical scene information with the highest scene similarity from the historical scene information according to the scene similarity matching algorithm as the target scene information. This can ensure that the selected target fatigue characteristics are similar to the scene information of the driver's fatigue characteristics, and the fatigue similarity determined under similar scene information is more accurate, thereby improving the reliability of current fatigue state recognition.

[0161] As an implementation manner of an embodiment of the present application, the above-mentioned current scene information and the above-mentioned historical scene information may both include scene information of at least one dimension.

[0162] The step of determining the scene similarity between the current scene information and the historical scene information according to the scene matching algorithm may include:

[0163] Determine the degree of difference between the scene information of each dimension included in the current scene information and the scene information of the dimension included in the historical scene information; and determine the scene similarity between the current scene information and the historical scene information based on the degree of difference corresponding to the scene information of each dimension.

[0164] Since the above-mentioned pre-established scene feature library can include the correspondence between historical scene information and historical fatigue features, the electronic device can calculate the degree of difference between the current scene information and each historical scene information in the scene feature library after acquiring the current scene information, and then determine the corresponding scene similarity based on the degree of difference. The greater the degree of difference between the current scene information and the historical scene information, the smaller the scene similarity between the previous scene information and the historical scene information; conversely, the smaller the degree of difference between the current scene information and the historical scene information, the greater the scene similarity between the previous scene information and the historical scene information.

[0165] In one embodiment, the current scene information and the historical scene information both include scene information of only one dimension. For example, the current scene information and the historical scene information both include only road types. Then, the degree of difference in road types between the road types in the current scene information and the historical road types in the historical scene information can be calculated. Then, the similarity of road types between the current scene information and the historical scene information can be calculated based on the degree of difference in road types, and the similarity can be used as the scene similarity between the current scene information and the historical scene information.

[0166] In one embodiment, the current scene information and the historical scene information include scene information of multiple dimensions. For example, the current scene information and the historical scene information include driving time, weather, and road type. Then, the time difference degree between the driving time in the current scene information and the historical driving time in the historical scene information, the weather difference degree between the weather in the current scene information and the historical weather in the historical scene information, and the road type difference degree between the road type in the current scene information and the historical road type in the historical scene information can be calculated respectively, and then the average difference degree of the time difference degree, the weather difference degree, and the road type difference degree can be calculated, and then the scene similarity between the current scene information and the historical scene information can be determined based on the average difference degree.

[0167] It can be seen that in this embodiment, the electronic device can determine the degree of difference between the scene information of each dimension included in the current scene information and the scene information of the dimension included in the historical scene information, and then determine the scene similarity between the current scene information and the historical scene information according to the degree of difference in multiple dimensions. In this way, the scene similarity between the current scene information and the historical scene information can be determined by scene information of multiple dimensions, and the included scene information is more comprehensive, so that the target scene information selected subsequently is closer to the current scene information.

[0168] As an implementation method of the present application, Figure 4As shown, the scene information of the at least one dimension includes at least one of driving time period, weather, road type, continuous driving time, and average eye distance;

[0169] The step of determining the degree of difference between the scene information of each dimension included in the current scene information and the scene information of the dimension included in the historical scene information may include:

[0170] S401, when the scenario information includes a driving period, determining the time difference degree between the current driving period and the historical driving period according to the minimum time difference between the current driving period included in the current scenario information and the historical driving period included in the historical scenario information.

[0171] When the driver drives the vehicle in the early morning or late at night, such as at 5 o'clock or 23 o'clock, the driver may not have enough rest and is prone to drowsiness, that is, driving the vehicle in a fatigued state. Therefore, the driver's driving time period can be obtained for subsequent determination of the fatigue state.

[0172] The driving period represents the period when the driver starts driving the vehicle. The time can be divided into 24 intervals according to the 24-hour system, or divided into 12 intervals according to the 12-hour system. No specific limitation is made here.

[0173] S402: When the scene information includes weather, determine the degree of weather difference between the current weather and the historical weather according to whether the current weather included in the current scene information is consistent with the historical weather included in the historical scene information.

[0174] The driving period represents the weather conditions of the driver's current scene, which may include: sunny, cloudy, rainy, snowy, foggy, etc.

[0175] Different weather conditions will also have an impact on the driver. For example, when it is sunny, the sunlight is quite dazzling and the driver is affected by the light and is not easily fatigued. When it is cloudy, rainy, snowy, foggy, etc., the ambient light will not affect the driver and the driver may be fatigued.

[0176] In addition, the weather conditions can be used as a reference and combined with scene information of other dimensions to identify the driver's status. For example, when the weather in the current scene information is foggy, it will affect the driver's visibility. In this case, if the driver is driving normally, the current speed should be slow, such as 20 km / h-40 km / h; if the current speed of the vehicle is determined to be very fast through the CAN signal, such as 60 km / h or more than 60 km / h, it may be because the driver is in a state of fatigue driving and accidentally stepped on the accelerator, resulting in a speed far greater than the theoretical speed under the current weather conditions. Based on this, the weather can be obtained for subsequent determination of fatigue status.

[0177] S403, when the scene information includes a road type, determining the degree of difference between the current road type and the historical road type according to whether the current road type included in the current scene information is consistent with the historical road type included in the historical scene information.

[0178] The road type represents the type of road the driver is currently driving the vehicle on, which may include: expressway, fast road, main road, secondary road, branch road, etc.

[0179] Since different road types are set with different speed ranges, the road type can be used as a reference and combined with other dimensions of scene information to identify the driver's status. For example, the satellite positioning system can be used to obtain that the driver is driving a vehicle on a highway, and the maximum speed of the highway is 120 kilometers per hour and the minimum speed is 60 kilometers per hour. If the current vehicle speed obtained through the CAN signal is less than 60 kilometers per hour, or greater than 120 kilometers per hour, it may be that the driver is in a fatigued state and his driving skills have declined, resulting in the current vehicle speed not being in the speed range of the current road type. Based on this, the road type can be obtained for subsequent determination of fatigue status.

[0180] S404, in the case where the scenario information includes continuous driving duration, determine the degree of difference between the current continuous driving duration and the historical continuous driving duration according to the minimum duration difference between the current continuous driving duration included in the current scenario information and the historical continuous driving duration included in the historical scenario information.

[0181] The continuous driving time represents the length of time the driver has been driving continuously without stopping from the start of driving the vehicle to the current period, and can be expressed in hours.

[0182] Continuous driving of the vehicle by the driver may cause the driver to have difficulty concentrating, have blurred vision, and be in a state of driving fatigue. Therefore, the continuous driving time can be obtained for determining the subsequent fatigue state.

[0183] S405, when the scene information includes the average eye distance, determine the degree of difference between the current average eye distance and the historical average eye distance according to the minimum average eye distance difference between the current average eye distance included in the current scene information and the historical average eye distance included in the historical scene information.

[0184] The average eye distance represents the average value of the distance between the driver's eyes from the start of the vehicle. The distance between the driver's upper eyelid and lower eyelid can be counted as the driver's eye distance according to a preset period, and the average of multiple eye distances can be calculated as the average eye distance. When the driver is in a fatigued state, the eyes are slightly closed or closed. Therefore, by determining the driver's average eye distance, it can be determined whether the driver's eyes are closed, and then whether the driver is in a fatigued driving state.

[0185] In the case where the scene information includes scene information of more than one dimension, the order in which the electronic device determines the degree of difference between the scene information of each dimension and the scene information of the corresponding dimension included in the historical scene information can be random, or the electronic device can simultaneously determine various degrees of difference corresponding to the scene information of multiple dimensions. In other words, the order of the above steps S401-S405 can be random and is not specifically limited here.

[0186] For example, when the scene information includes driving time period, weather, road type, continuous driving time and average eye distance, the electronic device can determine the degree of difference corresponding to each dimension in the order of determining the degree of difference in time period, the degree of difference in weather, the degree of difference in road type, the degree of difference in duration and the degree of difference in eye distance; it can also determine the degree of difference corresponding to each dimension in the order of determining the degree of difference in road type, the degree of difference in eye distance, the degree of difference in time period, the degree of difference in duration and the degree of difference in weather; it can also determine the degree of difference in time period, the degree of difference in weather, the degree of difference in road type, the degree of difference in duration and the degree of difference in eye distance at the same time.

[0187] The scene information including driving time period, continuous driving time and road type can be obtained through a driving recorder installed in the car or through a satellite positioning system; the scene information including weather can be obtained through a driving recorder, or a temperature sensor and a humidity sensor; the scene information including average eye distance can be obtained by analyzing images taken by an image acquisition device in the car. These are all reasonable and are not specifically limited here.

[0188] It can be seen that in this embodiment, the above five key fatigue factors: driving time, weather, road type, continuous driving time and average eye distance are used as scene information of different dimensions, and a scene feature library for fatigue driving recognition is established based on the above scene information of different dimensions. Then, in the scene feature library, the scene similarity matching method is used to effectively find a preset number of target scene information that is closest to the current driving scene information.

[0189] As an implementation of an embodiment of the present application, the step of determining the degree of time difference between the current driving period and the historical driving period according to the minimum time difference between the current driving period that can be included in the current scene information and the historical driving period that can be included in the historical scene information may include:

[0190] The difference between the current driving period and the historical driving period is calculated using the following formula: :

[0191] ;

[0192] Among them, the above is the current driving period, the above For historical driving periods.

[0193] The above step of determining the degree of weather difference between the current weather and the historical weather based on whether the current weather that may be included in the current scene information is consistent with the historical weather that may be included in the historical scene information may include:

[0194] Calculate the difference between the current weather and the historical weather using the following formula :

[0195] ;

[0196] Among them, the above For the current weather, the above For historical weather.

[0197] The step of determining the degree of difference between the current road type and the historical road type based on whether the current road type that can be included in the current scene information is consistent with the historical road type that can be included in the historical scene information may include:

[0198] The degree of road type difference between the current road type and the historical road type is calculated according to the following formula :

[0199] ;

[0200] Among them, the above is the current road type. It is a historical road type.

[0201] The step of determining the degree of difference between the current continuous driving time and the historical continuous driving time according to the minimum time difference between the current continuous driving time that can be included in the current scene information and the historical continuous driving time that can be included in the historical scene information may include:

[0202] The difference between the current continuous driving time and the historical continuous driving time is calculated using the following formula: :

[0203] ;

[0204] Among them, the above is the current continuous driving time, the above It is the historical continuous driving time.

[0205] The step of determining the degree of difference between the current average eye distance and the historical average eye distance according to the minimum average eye distance difference between the current average eye distance that may be included in the current scene information and the historical average eye distance that may be included in the historical scene information may include:

[0206] The difference between the current average eye distance and the historical average eye distance is calculated using the following formula :

[0207] ;

[0208] Among them, the above is the current average eye distance, the above The historical average eye distance.

[0209] As an implementation mode, when the above formula is used to calculate the time difference degree between the current driving period and the historical driving period, the current driving period and the historical driving period are both in 24-hour format, and the maximum value of the minimum time difference between the current driving period and the historical driving period is 12, so the above time difference degree ranges from 0 to 1. The smaller the difference between the current driving period and the historical driving period, the closer the calculated time difference degree is to 0; the larger the difference between the current driving period and the historical driving period, the closer the calculated time difference degree is to 1.

[0210] As an implementation method, when using the above formula to calculate the degree of weather difference between the current weather and the historical weather, the electronic device can determine whether the current weather is consistent with the historical weather. If the current weather is consistent with the historical weather, the degree of weather difference is confirmed to be 0; if the current weather is inconsistent with the historical weather, the degree of weather difference is confirmed to be 1.

[0211] As an implementation method, when using the above formula to calculate the degree of road type difference between the current road type and the historical road type, the electronic device can determine whether the current road type and the historical road type are consistent. If the current road type and the historical road type are consistent, the degree of road type difference is confirmed to be 0; if the current road type and the historical road type are inconsistent, the degree of road type difference is confirmed to be 1.

[0212] As an implementation mode, when the above formula is used to calculate the duration difference between the current continuous driving duration and the historical continuous driving duration, the maximum value of the minimum duration difference between the current continuous driving duration and the historical continuous driving duration is 12, so the above duration difference ranges from 0 to 1. The smaller the difference between the current continuous driving duration and the historical continuous driving duration, the closer the calculated time period difference is to 0; the larger the difference between the current continuous driving duration and the historical continuous driving duration, the closer the calculated time period difference is to 1.

[0213] As an implementation mode, when the above formula is used to calculate the difference between the current average eye distance and the historical average eye distance, the maximum value of the minimum difference between the current average eye distance and the historical average eye distance between the current continuous driving time and the historical continuous driving time is 0.3, so the range of the above eye distance difference is 0 to 1. The smaller the difference between the current average eye distance and the historical average eye distance, the closer the calculated eye distance difference is to 0; the larger the difference between the current average eye distance and the historical average eye distance, the closer the calculated eye distance difference is to 1.

[0214] It can be seen that in this embodiment, the electronic device can use the above formula to respectively calculate the degree of time difference between the current driving period and the historical driving period, the degree of weather difference between the current weather and the historical weather, the degree of road type difference between the current road type and the historical road type, the degree of time difference between the current continuous driving time and the historical continuous driving time, and the degree of eye distance difference between the current average eye distance and the historical average eye distance, and can accurately calculate the degree of difference corresponding to the scene information in each dimension mentioned above.

[0215] As an implementation of an embodiment of the present application, the step of determining the scene similarity between the current scene information and the historical scene information according to the degree of difference corresponding to the scene information in each dimension may include:

[0216] The scene similarity between the current scene information and the historical scene information is calculated according to the following formula :

[0217] ;

[0218] in, is the current scene information, For historical scene information, is the degree of time difference, is the degree of weather difference, is the degree of difference in road types, is the degree of time difference, The difference in eye distance.

[0219] When the current scene information and the historical scene information include driving time period, weather, road type, continuous driving time and average eye distance, the above formula is used to calculate the time period difference, weather difference, road type difference, time difference and eye distance difference between the current scene information and the historical scene information respectively, and then the above formula is used to calculate the scene similarity between the current scene information and the historical scene information based on the average value of the above difference degrees.

[0220] It can be seen that in this embodiment, the electronic device can use the above formula to calculate the scene similarity between the current scene information and the historical scene information, thereby improving the accuracy of the determined scene similarity.

[0221] As an implementation of an embodiment of the present application, the step of determining the fatigue similarity between the driver fatigue feature and each target fatigue feature may include:

[0222] Calculate the cosine similarity between the driver fatigue feature and each target fatigue feature; calculate the average of the cosine similarities as the fatigue similarity.

[0223] After selecting the TopN historical scene information with the highest similarity in the scene feature library as the target scene information, the electronic device can determine the TopN target fatigue feature sequences according to the correspondence between the historical scene information and the target scene information, and then calculate the cosine similarity between the driver fatigue feature and the target fatigue features from Top1 to TopN in the TopN target fatigue feature sequences. After obtaining N cosine similarities, the average of these N cosine similarities is calculated as the fatigue similarity.

[0224] It can be seen that in this embodiment, the electronic device can accurately determine the similarity between the driver's fatigue characteristics and the target fatigue characteristics by calculating the cosine similarity between the driver's fatigue characteristics and each target fatigue characteristic, and taking the average value of the above cosine similarities as the fatigue similarity value, thereby facilitating the subsequent confirmation of the fatigue state of the driver to be identified based on the similarity.

[0225] As an implementation of an embodiment of the present application, the step of determining the fatigue state of the driver to be identified based on the fatigue similarity may include:

[0226] Based on the difference between the fatigue similarity and the preset value, it is determined that the fatigue state of the driver to be identified is fatigue driving or normal driving.

[0227] The above preset value is used to characterize the fatigue similarity corresponding to fatigue driving or normal driving. Since the historical fatigue features stored in the scene feature library are the features of the driver in a fatigued state, when the value of the fatigue similarity calculated by the target fatigue feature is closer to 1, it means that the difference between the fatigue similarity and the preset value is smaller, and the fatigue state of the driver to be identified is fatigue driving; conversely, when the value of the fatigue similarity calculated by the target fatigue feature is closer to 0, it means that the difference between the fatigue similarity and the preset value is greater, and the fatigue state of the driver to be identified is normal driving.

[0228] As an implementation mode, when it is determined that the fatigue state of the driver to be identified is fatigue driving, the electronic device can send an alarm signal to remind the driver who is in fatigue driving, thereby reducing the occurrence of traffic accidents caused by fatigue driving.

[0229] It can be seen that in this embodiment, the electronic device can accurately identify whether the fatigue state of the driver to be identified is fatigue driving or normal driving based on the difference between the fatigue similarity and the preset value.

[0230] As an implementation of the embodiment of the present application, the step of obtaining the driving characteristics of the driver to be identified may include:

[0231] During the process of the driver to be identified driving the vehicle, an in-vehicle image captured by the in-vehicle image sensor within a preset time period is obtained, and image recognition is performed on the in-vehicle image to obtain at least one of the eye opening degree, mouth opening degree, sight direction angle, head posture angle and human body key points of the driver to be identified as an in-vehicle parameter; an outside-vehicle image captured by the outside-vehicle image sensor within the preset time period is obtained, and image recognition is performed on the outside-vehicle image to obtain at least one of the lane offset distance, the number of surrounding vehicles and the front vehicle collision distance as an outside-vehicle parameter; at least one of the vehicle speed, acceleration and steering wheel angle of the vehicle within the preset time period is obtained as a driving parameter; the in-vehicle parameters, the outside-vehicle parameters and the driving parameters are combined into a feature vector of a preset dimension according to the acquisition time to obtain the driving characteristics of the driver to be identified.

[0232] When acquiring the driving characteristics of the driver to be identified, the electronic device can perform image recognition on the in-vehicle images collected by the in-vehicle image sensor to obtain at least one of the eye opening degree, mouth opening degree, sight direction angle, head posture angle and human body key points of the driver to be identified as in-vehicle parameters; perform image recognition on the outside-vehicle images collected by the outside-vehicle image sensor to obtain at least one of the lane offset distance, the number of surrounding vehicles and the front vehicle collision distance as out-vehicle parameters, and obtain at least one of the vehicle speed, acceleration and steering wheel angle as driving parameters.

[0233] After obtaining the above-mentioned in-vehicle parameters, out-of-vehicle parameters and driving parameters, the three types of parameters can be combined into a preset feature vector according to the time when the above-mentioned parameters are obtained, that is, the driving characteristics of the driver to be identified are obtained.

[0234] It can be seen that in this embodiment, the electronic device uses the acquired in-vehicle parameters, out-of-vehicle parameters and driving parameters to form a feature vector of preset dimensions according to time as the driving characteristics of the driver to be identified. Based on this, the driving characteristics of the driver to be identified acquired by the electronic device contain key information for fatigue state identification. Since the above driving characteristics contain three types of different parameter information, the fatigue characteristics of the driver to be identified can be identified more comprehensively.

[0235] In order to facilitate understanding of the method provided by the present application, the following Figure 5 The specific embodiment shown is described below:

[0236] The electronic device can record the recognition signal of the in-vehicle image sensor, the recognition signal of the out-vehicle image sensor, the CAN signal and the current scene information in a manner of recording for N seconds and collecting M frames per second to obtain signal sequence 1, signal sequence 2...signal sequence N for fatigue state recognition, wherein the input length of each of the above signals is N×M; signal sequence 1 to signal sequence N can be respectively the degree of eye opening, the degree of mouth opening, the direction angle of sight, the head posture angle, the key points of the human body, the lane deviation distance, the number of surrounding vehicles, the collision distance of the front vehicle, the vehicle speed, the acceleration and the steering wheel angle.

[0237] After acquiring the above N signal sequences, the signal sequences are sampled based on the time series to obtain driving features with a certain dimension, and then the above driving features are input into the neural network model, and fatigue hidden features are obtained through convolution operations of the neural network model. In the case where the electronic device is used to infer the fatigue state of the driver to be identified, the neural network model is a pre-trained fatigue feature extraction model, and the fatigue hidden features are the driver fatigue features; in the case where the electronic device is used to train the fatigue feature extraction model, the neural network model is an initial fatigue feature extraction model, and the fatigue hidden features are predicted fatigue features.

[0238] The electronic device inputs the acquired current scene information into a pre-established scene feature library, and selects N historical scene information with the highest similarity from the historical scene information in the scene feature library as target scene information according to the scene matching algorithm. These N target scene information can be marked as Top1 target scene information, Top2 target scene information...TopN target scene information according to the degree of similarity. Then, based on the correspondence between the historical scene information and the historical fatigue features included in the scene feature library, the Top1 target fatigue features, Top2 target fatigue features...TopN target fatigue features corresponding to the Top1 target scene information, Top2 target scene information...TopN target scene information are determined respectively.

[0239] In the case where the electronic device is used to infer the fatigue state of the driver to be identified, the fatigue similarities between the fatigue hidden features, i.e., the driver fatigue features and the Top1 target fatigue features, the Top2 target fatigue features...TopN target fatigue features are calculated in sequence according to the similarity algorithm, thereby obtaining similarities S1, S2, S3...Sn, and then the average value of the above n similarities is calculated as the fatigue similarity, thereby determining the fatigue state of the driver to be identified based on the fatigue similarity.

[0240] In the case where the electronic device is used to train the fatigue feature extraction model, after obtaining the hidden fatigue features output by the neural network model, that is, the initial fatigue feature extraction model, that is, the predicted fatigue features, the predicted fatigue features are input into the fully connected network connected to the neural network model, and the predicted fatigue features are further extracted by the fully connected network to obtain a one-dimensional fatigue feature vector corresponding to the predicted fatigue features. Next, the one-dimensional fatigue feature vector is mapped using an activation function to obtain an estimated fatigue confidence corresponding to the one-dimensional fatigue feature vector, and then the loss function is used to calculate the loss value between the estimated fatigue confidence and the corresponding true value of the fatigue confidence, and then the parameters of the convolution layer are adjusted based on the loss value until the initial fatigue feature extraction model converges to obtain the fatigue feature extraction model.

[0241] In this way, the electronic device makes full use of the signals of each sensor and the scene information of the vehicle. Compared with the fatigue state recognition method based on a single sensor, the embodiment of the present application has higher reliability and generalization. Based on the fatigue state recognition method, it is no longer a simple algorithm judgment based on the current instantaneous behavior, but a comprehensive judgment that combines the current scene information and the historical scene information of the scene feature library, as well as the combination of online data, i.e., driver fatigue characteristics and offline data (historical fatigue characteristics), which theoretically improves the upper limit of fatigue state recognition.

[0242] In the technical solution of this application, the operations involved in obtaining, storing, using, processing, transmitting, providing and disclosing user personal information are all carried out with the user's authorization.

[0243] Corresponding to the above fatigue state recognition method, the present application embodiment further provides a fatigue state recognition device. The following is an introduction to the fatigue state recognition device provided by the present application embodiment.

[0244] like Figure 6 As shown, a fatigue state recognition device comprises:

[0245] The feature information acquisition module 601 is used to acquire the driving features and current scene information of the driver to be identified;

[0246] The driving characteristics represent the state of the driver to be identified and the driving state of the vehicle.

[0247] A fatigue feature acquisition module 602 is used to input the driving feature into a pre-trained fatigue feature extraction model for convolution processing to obtain the driver fatigue feature output by the fatigue feature extraction model;

[0248] Wherein, the fatigue feature extraction model includes multiple convolutional layers.

[0249] The target scene information determination module 603 is used to determine a preset number of historical scene information matching the current scene information from the historical scene information in the pre-established scene feature library according to the scene matching algorithm as the target scene information;

[0250] The scene feature library includes the correspondence between historical scene information and historical fatigue features.

[0251] The fatigue similarity determination module 604 is used to determine the target fatigue feature corresponding to the target scene information according to the corresponding relationship, and determine the fatigue similarity between the driver fatigue feature and each target fatigue feature.

[0252] The fatigue state determination module 605 is used to determine the fatigue state of the driver to be identified based on the fatigue similarity.

[0253] It can be seen that in the solution provided by the embodiment of the present application, the electronic device can obtain the driving characteristics and current scene information of the driver to be identified, wherein the driving characteristics represent the state of the driver to be identified and the driving state of the vehicle, and then input the driving characteristics into the pre-trained fatigue feature extraction model for convolution processing to obtain the driver fatigue characteristics output by the fatigue feature extraction model, wherein the fatigue feature extraction model includes multiple convolution layers; then from the historical scene information of the pre-established scene feature library, according to the scene matching algorithm, a preset number of historical scene information matching the current scene information is determined as the target scene information, wherein the scene feature library includes the corresponding relationship between the historical scene information and the historical fatigue characteristics; then according to the corresponding relationship, the target fatigue characteristics corresponding to the target scene information are determined, and the fatigue similarity between the driver fatigue characteristics and each target fatigue characteristic is determined, so as to determine the fatigue state of the driver to be identified based on the fatigue similarity. Since the fatigue states corresponding to the same scene have similarities, the current scene information is matched with the historical scene information of the pre-established scene feature library, so that the determined target fatigue characteristics are capable of representing the characteristics of the fatigue state under the current scene information. Then, based on the fatigue similarity between the driver's fatigue characteristics and each target fatigue characteristic, the fatigue state of the driver to be identified can be obtained, and the driver's fatigue state can be accurately determined. The fatigue degree of the driver to be identified is determined through information from multiple dimensions, thereby improving the accuracy of identifying the fatigue state.

[0254] As an implementation of the embodiment of the present application, the device fatigue feature acquisition module 602 may include:

[0255] The sample information acquisition submodule is used to obtain driving feature samples and the fatigue confidence truth value corresponding to each driving feature sample.

[0256] The predicted fatigue feature acquisition submodule is used to input the driving feature samples into the initial fatigue feature extraction model so that the multiple convolution layers included in the initial fatigue feature extraction model perform convolution processing on the driving feature samples and output the obtained predicted fatigue features to the fully connected network.

[0257] The one-dimensional fatigue feature vector acquisition submodule is used to process the predicted fatigue feature with a fully connected network to obtain a one-dimensional fatigue feature vector corresponding to the predicted fatigue feature.

[0258] The estimated fatigue confidence determination submodule is used to map the one-dimensional fatigue feature vector using an activation function to obtain the estimated fatigue confidence corresponding to the one-dimensional fatigue feature vector.

[0259] The loss value calculation submodule is used to calculate the loss value between the estimated fatigue confidence and the corresponding fatigue confidence true value using the loss function.

[0260] The parameter adjustment submodule is used to adjust the parameters of the convolution layer based on the loss value until the initial fatigue feature extraction model converges to obtain the fatigue feature extraction model.

[0261] As an implementation of the embodiment of the present application, the above-mentioned device may also include:

[0262] The historical driving feature acquisition module is used to acquire historical scene information and the driving features of the driver acquired in the scene corresponding to the historical scene information as historical driving features.

[0263] The historical fatigue feature determination module is used to input the historical driving features into a pre-trained fatigue feature extraction model for convolution processing to obtain the historical fatigue features output by the fatigue feature extraction model.

[0264] The scene feature library establishment module is used to establish and store the corresponding relationship between historical scene information and historical fatigue features to obtain a scene feature library.

[0265] As an implementation of the embodiment of the present application, the target scene information determination module 603 may include:

[0266] The scene similarity determination submodule is used to determine the scene similarity between the current scene information and the historical scene information according to the scene matching algorithm for each historical scene information included in the scene feature library.

[0267] The target scene information selection submodule is used to select a preset number of historical scene information with the highest corresponding scene similarity from the historical scene information as the target scene information.

[0268] As an implementation manner of an embodiment of the present application, the current scene information and the historical scene information may include scene information of at least one dimension; and the scene similarity determination submodule may include:

[0269] The difference degree determination unit is used to determine the difference degree between the scene information of each dimension included in the current scene information and the scene information of the dimension included in the historical scene information.

[0270] The scene similarity determination unit is used to determine the scene similarity between the current scene information and the historical scene information according to the degree of difference corresponding to the scene information of each dimension.

[0271] As an implementation of an embodiment of the present application, the scene information of the at least one dimension may include at least one of driving time period, weather, road type, continuous driving time and average eye distance; the difference degree determination unit may include:

[0272] a time period difference degree determination subunit, for determining the time period difference degree between the current driving period and the historical driving period according to the minimum time difference between the current driving period included in the current scene information and the historical driving period included in the historical scene information, when the scene information includes the driving period;

[0273] a weather difference degree determination subunit, for determining the weather difference degree between the current weather and the historical weather according to whether the current weather included in the current scene information is consistent with the historical weather included in the historical scene information, when the scene information includes the weather;

[0274] a road type difference degree determination subunit, for determining the road type difference degree between the current road type and the historical road type according to whether the current road type included in the current scene information is consistent with the historical road type included in the historical scene information, when the scene information includes the road type;

[0275] a duration difference degree determination subunit, for determining the duration difference degree between the current continuous driving duration and the historical continuous driving duration according to the minimum duration difference between the current continuous driving duration included in the current scenario information and the historical continuous driving duration included in the historical scenario information, when the scenario information includes the driving duration;

[0276] The eye distance difference degree determination subunit is used to determine the eye distance difference degree between the current average eye distance and the historical average eye distance according to the minimum average eye distance difference between the current average eye distance included in the current scene information and the historical average eye distance included in the historical scene information when the scene information includes the average eye distance.

[0277] As an implementation of an embodiment of the present application, the above-mentioned time period difference degree determination subunit can be specifically used for:

[0278] The degree of time difference between the current driving period and the historical driving period is calculated according to the following formula :

[0279] ;

[0280] Among them, the above is the current driving period, the above For historical driving periods.

[0281] The weather difference degree determination subunit can be specifically used for:

[0282] Calculate the degree of difference between the current weather and the historical weather using the following formula :

[0283] ;

[0284] Among them, the above For the current weather, the above For historical weather.

[0285] The road type difference degree determination subunit can be specifically used for:

[0286] The degree of difference between the current road type and the historical road type is calculated according to the following formula: :

[0287] ;

[0288] Among them, the above is the current road type. It is a historical road type.

[0289] The duration difference degree determination subunit may be specifically used for:

[0290] The difference between the current continuous driving time and the historical continuous driving time is calculated using the following formula: :

[0291] ;

[0292] Among them, the above is the current continuous driving time, the above It is the historical continuous driving time.

[0293] The above-mentioned eye distance difference degree determination subunit can be specifically used for:

[0294] The difference between the current average eye distance and the historical average eye distance is calculated using the following formula: :

[0295] ;

[0296] Among them, the above is the current average eye distance, the above The historical average eye distance.

[0297] As an implementation of an embodiment of the present application, the above-mentioned scene similarity determination unit can be specifically used for:

[0298] The scene similarity between the current scene information and the historical scene information is calculated according to the following formula :

[0299] ;

[0300] in, is the current scene information, For historical scene information, is the degree of time difference, is the degree of weather difference, is the degree of difference in road types, is the degree of time difference, The difference in eye distance.

[0301] As an implementation of the embodiment of the present application, the fatigue similarity determination module 604 may include:

[0302] The cosine similarity calculation submodule is used to calculate the cosine similarity between the driver fatigue feature and each target fatigue feature.

[0303] The fatigue similarity calculation submodule is used to calculate the average value of the cosine similarity as the fatigue similarity.

[0304] As an implementation of the embodiment of the present application, the fatigue state determination module 605 may include:

[0305] A fatigue state determination submodule, used to determine whether the fatigue state of the driver to be identified is fatigue driving or normal driving based on the difference between the fatigue similarity and a preset value;

[0306] The preset value is used to characterize the fatigue similarity corresponding to fatigue driving or normal driving.

[0307] As an implementation of the embodiment of the present application, the feature information acquisition module 601 may include:

[0308] The in-vehicle parameter acquisition submodule is used to obtain the in-vehicle image collected by the in-vehicle image sensor within a preset time period during the process of the driver to be identified driving the vehicle, and perform image recognition on the in-vehicle image to obtain at least one of the eye opening degree, mouth opening degree, line of sight direction angle, head posture angle and human body key points of the driver to be identified as the in-vehicle parameter.

[0309] The vehicle exterior parameter acquisition submodule is used to acquire the vehicle exterior image captured by the vehicle exterior image sensor within a preset time period, and perform image recognition on the vehicle exterior image to obtain at least one of the lane offset distance, the number of surrounding vehicles, and the front vehicle collision distance as the vehicle exterior parameter.

[0310] The driving parameter acquisition submodule is used to acquire at least one of the vehicle speed, acceleration and steering wheel angle of the vehicle within a preset time period as a driving parameter.

[0311] The driving characteristics determination submodule is used to combine the in-vehicle parameters, the out-vehicle parameters and the driving parameters into a feature vector of preset dimensions according to the acquisition time, so as to obtain the driving characteristics of the driver to be identified.

[0312] The present application also provides an electronic device, such as Figure 7 As shown, including:

[0313] Memory 701, used for storing computer programs;

[0314] The processor 702 is used to implement the fatigue state recognition method described in any of the above embodiments when executing the program stored in the memory 701.

[0315] It can be seen that in the solution provided by the embodiment of the present application, the electronic device can obtain the driving characteristics and current scene information of the driver to be identified, wherein the driving characteristics represent the state of the driver to be identified and the driving state of the vehicle, and then input the driving characteristics into the pre-trained fatigue feature extraction model for convolution processing to obtain the driver fatigue characteristics output by the fatigue feature extraction model, wherein the fatigue feature extraction model includes multiple convolution layers; then from the historical scene information of the pre-established scene feature library, according to the scene matching algorithm, a preset number of historical scene information matching the current scene information is determined as the target scene information, wherein the scene feature library includes the corresponding relationship between the historical scene information and the historical fatigue characteristics; then according to the corresponding relationship, the target fatigue characteristics corresponding to the target scene information are determined, and the fatigue similarity between the driver fatigue characteristics and each target fatigue characteristic is determined, so as to determine the fatigue state of the driver to be identified based on the fatigue similarity. Since the fatigue states corresponding to the same scene have similarities, the current scene information is matched with the historical scene information of the pre-established scene feature library, so that the determined target fatigue characteristics are capable of representing the characteristics of the fatigue state under the current scene information. Then, based on the fatigue similarity between the driver's fatigue characteristics and each target fatigue characteristic, the fatigue state of the driver to be identified can be obtained, and the driver's fatigue state can be accurately determined. The fatigue degree of the driver to be identified is determined through information from multiple dimensions, thereby improving the accuracy of identifying the fatigue state.

[0316] Furthermore, the electronic device may further include a communication bus and / or a communication interface, and the processor 702, the communication interface, and the memory 701 communicate with each other via the communication bus.

[0317] The communication bus mentioned in the above electronic device can be a Peripheral Component Interconnect (PCI) bus or an Extended Industry Standard Architecture (EISA) bus, etc. The communication bus can be divided into an address bus, a data bus, a control bus, etc. For ease of representation, only one thick line is used in the figure, but it does not mean that there is only one bus or one type of bus.

[0318] The communication interface is used for communication between the above electronic device and other devices.

[0319] The memory may include a random access memory (RAM) or a non-volatile memory (NVM), such as at least one disk memory. Optionally, the memory may also be at least one storage device located away from the aforementioned processor.

[0320] The above-mentioned processor can be a general-purpose processor, including a central processing unit (CPU), a network processor (NP), etc.; it can also be a digital signal processor (DSP), an application specific integrated circuit (ASIC), a field programmable gate array (FPGA) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components.

[0321] In another embodiment provided in the present application, a computer-readable storage medium is provided, in which a computer program is stored. When the computer program is executed by a processor, the steps of any of the above fatigue state identification methods are implemented.

[0322] In another embodiment provided in the present application, a computer program product including instructions is also provided, which, when executed on a computer, enables the computer to execute any fatigue state identification method in the above embodiments.

[0323] In the above embodiments, it can be implemented in whole or in part by software, hardware, firmware or any combination thereof. When implemented using software, it can be implemented in whole or in part in the form of a computer program product. The computer program product includes one or more computer instructions. When the computer program instructions are loaded and executed on a computer, the process or function described in the embodiment of the present application is generated in whole or in part. The computer may be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device. The computer instructions may be stored in a computer-readable storage medium, or transmitted from one computer-readable storage medium to another computer-readable storage medium. For example, the computer instructions may be transmitted from a website site, computer, server or data center to another website site, computer, server or data center by wired (e.g., coaxial cable, optical fiber, digital subscriber line (DSL)) or wireless (e.g., infrared, wireless, microwave, etc.) means. The computer-readable storage medium may be any available medium that a computer can access or a data storage device such as a server or data center that includes one or more available media integrated. The available medium may be a magnetic medium (e.g., a floppy disk, a hard disk, a tape), an optical medium (e.g., a DVD), or a solid-state hard disk (SSD), etc.

[0324] It should be noted that, in this article, relational terms such as first and second, etc. are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Moreover, the terms "include", "comprise" or any other variants thereof are intended to cover non-exclusive inclusion, so that a process, method, article or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or device. In the absence of further restrictions, the elements defined by the sentence "comprise a ..." do not exclude the existence of other identical elements in the process, method, article or device including the elements.

[0325] Each embodiment in this specification is described in a related manner, and the same or similar parts between the embodiments can be referred to each other, and each embodiment focuses on the differences from other embodiments. In particular, for the device, electronic device, computer-readable storage medium, and computer program product embodiments, since they are basically similar to the method embodiments, the description is relatively simple, and the relevant parts can be referred to the partial description of the method embodiments.

[0326] The above description is only a preferred embodiment of the present application and is not intended to limit the protection scope of the present application. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present application are included in the protection scope of the present application.

Claims

1. A method for identifying a fatigue state, characterized in that: The method comprises: Acquiring driving characteristics of the driver to be identified and current scene information, wherein the driving characteristics represent the state of the driver to be identified and the driving state of the vehicle; Inputting the driving feature into a pre-trained fatigue feature extraction model for convolution processing to obtain the driver fatigue feature output by the fatigue feature extraction model, wherein the fatigue feature extraction model includes a plurality of convolution layers; Determining, from historical scene information in a pre-established scene feature library, a preset number of historical scene information matching the current scene information according to a scene matching algorithm as target scene information, wherein the scene feature library includes a correspondence between historical scene information and historical fatigue features; According to the corresponding relationship, determining the target fatigue feature corresponding to the target scene information, and determining the fatigue similarity between the driver fatigue feature and each target fatigue feature; Based on the fatigue similarity, the fatigue state of the driver to be identified is determined.

2. The method according to claim 1, characterized in that The training method of the fatigue feature extraction model includes: Obtain driving feature samples and fatigue confidence truth values ​​corresponding to each driving feature sample; Inputting the driving feature sample into an initial fatigue feature extraction model so that a plurality of convolutional layers included in the initial fatigue feature extraction model perform convolution processing on the driving feature sample and outputting the obtained predicted fatigue feature to a fully connected network; The fully connected network processes the predicted fatigue feature to obtain a one-dimensional fatigue feature vector corresponding to the predicted fatigue feature; Mapping the one-dimensional fatigue feature vector using an activation function to obtain an estimated fatigue confidence corresponding to the one-dimensional fatigue feature vector; Calculating the loss value between the estimated fatigue confidence and the corresponding fatigue confidence true value using a loss function; The parameters of the convolutional layer are adjusted based on the loss value until the initial fatigue feature extraction model converges to obtain the fatigue feature extraction model.

3. The method according to claim 1, characterized in that Before the step of determining, from the historical scene information in the pre-established scene feature library, a preset number of historical scene information matching the current scene information according to a scene matching algorithm, the method further comprises: Acquire historical scene information and a driving characteristic of a driver acquired in a scene corresponding to the historical scene information as a historical driving characteristic; Inputting the historical driving features into a pre-trained fatigue feature extraction model for convolution processing to obtain historical fatigue features output by the fatigue feature extraction model; A corresponding relationship between the historical scene information and the historical fatigue features is established and stored to obtain a scene feature library.

4. The method according to claim 1, characterized in that: The step of determining, from the historical scene information in the pre-established scene feature library, a preset number of historical scene information matching the current scene information according to a scene matching algorithm as target scene information comprises: For each historical scene information included in the scene feature library, determining the scene similarity between the current scene information and the historical scene information according to a scene matching algorithm; A preset number of historical scene information with the highest corresponding scene similarities are selected from the historical scene information as target scene information.

5. The method according to claim 4, characterized in that The current scene information and the historical scene information both include scene information of at least one dimension; The step of determining the scene similarity between the current scene information and the historical scene information according to the scene matching algorithm comprises: Determining a degree of difference between the scene information of each dimension included in the current scene information and the scene information of the dimension included in the historical scene information; The scene similarity between the current scene information and the historical scene information is determined according to the degree of difference corresponding to the scene information of each dimension.

6. The method according to claim 5, characterized in that The at least one dimension of scene information includes at least one of driving time period, weather, road type, continuous driving time, and average eye distance; The step of determining the degree of difference between the scene information of each dimension included in the current scene information and the scene information of the dimension included in the historical scene information comprises: In the case where the scenario information includes a driving period, determining a time difference degree between the current driving period and the historical driving period according to a minimum time difference between a current driving period included in the current scenario information and a historical driving period included in the historical scenario information; In the case where the scene information includes weather, determining the degree of weather difference between the current weather and the historical weather according to whether the current weather included in the current scene information is consistent with the historical weather included in the historical scene information; In the case where the scene information includes a road type, determining a degree of difference in road type between the current road type and the historical road type according to whether the current road type included in the current scene information is consistent with the historical road type included in the historical scene information; In the case where the scenario information includes continuous driving duration, determining the duration difference between the current continuous driving duration and the historical continuous driving duration according to the minimum duration difference between the current continuous driving duration included in the current scenario information and the historical continuous driving duration included in the historical scenario information; In the case where the scene information includes the average eye distance, the degree of difference in the eye distance between the current average eye distance and the historical average eye distance is determined according to the minimum average eye distance difference between the current average eye distance included in the current scene information and the historical average eye distance included in the historical scene information.

7. The method according to claim 6, characterized in that The step of determining the degree of time difference between the current driving period and the historical driving period according to the minimum time difference between the current driving period included in the current scenario information and the historical driving period included in the historical scenario information comprises: The difference between the current driving period and the historical driving period is calculated according to the following formula: : ； Among them, the is the current driving period, For historical driving time; The step of determining the degree of weather difference between the current weather and the historical weather according to whether the current weather included in the current scene information is consistent with the historical weather included in the historical scene information comprises: The degree of weather difference between the current weather and the historical weather is calculated according to the following formula : ； Among them, the For the current weather, For historical weather; The step of determining the degree of difference between the current road type and the historical road type according to whether the current road type included in the current scene information is consistent with the historical road type included in the historical scene information comprises: The degree of difference between the current road type and the historical road type is calculated according to the following formula : ； Among them, the is the current road type, It is a historical road type; The step of determining the degree of difference between the current continuous driving duration and the historical continuous driving duration according to the minimum duration difference between the current continuous driving duration included in the current scene information and the historical continuous driving duration included in the historical scene information comprises: The difference between the current continuous driving duration and the historical continuous driving duration is calculated according to the following formula: : ； Among them, the is the current continuous driving time, The historical continuous driving time; The step of determining the degree of difference between the current average eye distance and the historical average eye distance according to the minimum average eye distance difference between the current average eye distance included in the current scene information and the historical average eye distance included in the historical scene information comprises: The difference between the current average eye distance and the historical average eye distance is calculated according to the following formula: : ； Among them, the is the current average eye distance, The historical average eye distance.

8. The method according to claim 7, characterized in that The step of determining the scene similarity between the current scene information and the historical scene information according to the degree of difference corresponding to the scene information of each dimension includes: The scene similarity between the current scene information and the historical scene information is calculated according to the following formula : ； Among them, the is the current scene information, is the historical scene information, is the degree of difference between the time periods, is the degree of weather difference, is the degree of difference of the road types, is the duration difference, is the degree of difference in interocular distance.

9. The method according to any one of claims 1 to 8, characterized in that: The step of determining the fatigue similarity between the driver fatigue feature and each target fatigue feature comprises: Calculating the cosine similarity between the driver fatigue feature and each target fatigue feature; The average value of the cosine similarities is calculated as the fatigue similarity.

10. The method according to claim 9, characterized in that The step of determining the fatigue state of the driver to be identified based on the fatigue similarity comprises: Based on the difference between the fatigue similarity and a preset value, it is determined that the fatigue state of the driver to be identified is fatigue driving or normal driving, wherein the preset value is used to characterize the fatigue similarity corresponding to fatigue driving or normal driving.

11. The method according to any one of claims 1 to 8, characterized in that: The step of acquiring the driving characteristics of the driver to be identified comprises: When the driver to be identified is driving the vehicle, an in-vehicle image captured by an in-vehicle image sensor within a preset time period is obtained, and image recognition is performed on the in-vehicle image to obtain at least one of the eye opening degree, mouth opening degree, sight direction angle, head posture angle, and human body key points of the driver to be identified as an in-vehicle parameter; Acquire an image outside the vehicle collected by an image sensor outside the vehicle within the preset time period, and perform image recognition on the image outside the vehicle to obtain at least one of a lane deviation distance, a number of surrounding vehicles, and a front vehicle collision distance as an external vehicle parameter; Acquiring at least one of the vehicle speed, acceleration and steering wheel angle of the vehicle within the preset time period as a driving parameter; The in-vehicle parameters, the out-vehicle parameters and the driving parameters are combined into a feature vector of a preset dimension according to the acquisition time to obtain the driving characteristics of the driver to be identified.

12. A fatigue state recognition device, characterized in that: The device comprises: A feature information acquisition module, used to acquire driving features of the driver to be identified and current scene information, wherein the driving features represent the state of the driver to be identified and the driving state of the vehicle; A fatigue feature acquisition module, used for inputting the driving feature into a pre-trained fatigue feature extraction model for convolution processing to obtain the driver fatigue feature output by the fatigue feature extraction model, wherein the fatigue feature extraction model includes a plurality of convolution layers; A target scene information determination module, used to determine a preset number of historical scene information matching the current scene information from historical scene information in a pre-established scene feature library according to a scene matching algorithm as target scene information, wherein the scene feature library includes a correspondence between historical scene information and historical fatigue features; a fatigue similarity determination module, configured to determine, based on the corresponding relationship, a target fatigue feature corresponding to the target scene information, and determine a fatigue similarity between the driver fatigue feature and each target fatigue feature; A fatigue state determination module is used to determine the fatigue state of the driver to be identified based on the fatigue similarity.

Images