Facial recognition method and system based on fatigue driving

Abstract

The invention provides a facial recognition method and system based on fatigue driving, and the method comprises the steps: obtaining a historical facial image of a user, recognizing facial features according to the historical facial image, and enabling the facial features to at least comprise eye features, eyebrow features and mouth features; respectively establishing independent neural network models of the eye features, the eyebrow features and the mouth features and a composite neural network model with a combination relationship, and training the independent neural network models and the composite neural network model by using the corresponding historical facial features; in a vehicle driving process, acquiring a face image of a user in real time, determining unshielded target features, inputting the target features into the corresponding trained neural network model, and outputting a score of fatigue driving; and according to the score of the fatigue driving, outputting a fatigue driving detection result, specifically, through the method, when the shielding object exists on the face, the fatigue driving detection can still be carried out, and the detection sensitivity is ensured.

Description

Technical Field

[0001] The present invention belongs to the technical field of facial recognition, and in particular relates to a facial recognition method and system based on fatigue driving. Background Art

[0002] Facial recognition technology is a biometric recognition technology based on artificial intelligence. It collects facial images or video streams through cameras and other devices, and uses deep learning algorithms to extract key facial feature points, such as the shape, position and relative distance of eyes, nose, mouth, etc., to form a unique facial feature vector, which is then compared and matched with the templates stored in the database to achieve identity authentication, access control, security monitoring, payment authentication and other functions. It is widely used in finance, transportation, public security and other fields.

[0003] At present, facial recognition technology has been used in some fatigue driving detections. It mainly identifies facial features and determines whether fatigue driving exists based on the recognition results. If fatigue driving exists, timely warnings or alarms are issued.

[0004] However, some drivers wear sunglasses, masks and other sunshades to block sunlight while driving. Since key facial features cannot be recognized, the facial recognition effect is greatly reduced, which is not conducive to fatigue driving detection. Summary of the Invention

[0005] Based on this, an embodiment of the present invention provides a facial recognition method and system based on fatigue driving, which aims to ensure the sensitivity of fatigue driving detection when there are obstructions on the face.

[0006] A first aspect of an embodiment of the present invention provides a facial recognition method based on fatigue driving, the method comprising: Obtaining a historical facial image of the user, and identifying facial features based on the historical facial image, the facial features including at least eye features, eyebrow features, and mouth features; Establishing independent neural network models for eye features, eyebrow features, and mouth features, and a composite neural network model with a combination relationship, and training the independent neural network models and the composite neural network model using the corresponding historical facial features; During vehicle driving, a facial image of the user is acquired in real time, unobstructed target features are determined, and the target features are input into the corresponding trained independent neural network model or the composite neural network model to output a fatigue driving score; According to the fatigue driving score, the fatigue driving detection result is output.

[0007] Furthermore, the method further comprises: According to the target features and the number of target features, a neural network model with different inputs and the same output is established and trained. The fewer the number of target features, the more types of inputs the neural network model can have.

[0008] Furthermore, in the step of establishing and training a neural network model with different inputs and the same output according to the target features and the number of target features, Establishing a first neural network model of the eyebrow feature, the first feature, and the second feature, as well as the mouth feature, the first feature, and the second feature; Establishing a second neural network model of eyebrow features, mouth features and the first feature; Among them, the first feature is the in-car atmosphere feature or the driving feature. When the first feature is the in-car atmosphere feature, the second feature is the driving feature. When the first feature is the driving feature, the second feature is the in-car atmosphere feature. The in-car atmosphere feature is used to characterize the sleeping environment in the car, and the driving feature characterizes the user's driving condition.

[0009] Furthermore, the in-car atmosphere characteristics include at least in-car music characteristics, communication characteristics, and temperature characteristics. The in-car music characteristics include at least volume sub-features and melody sub-features, and the communication characteristics include at least decibel sub-features and emotion sub-features.

[0010] Furthermore, the driving characteristics include at least vehicle speed sub-characteristics, steering wheel operation sub-characteristics, and accelerator / brake pedal operation sub-characteristics.

[0011] Furthermore, the neural network model is composed of an input layer, a hidden layer, and an output layer. The input layer includes a plurality of input nodes, the number of which is the same as the number of input data types, and is used to receive different types of input data. The hidden layer consists of five sublayers. The first hidden sublayer has 64 nodes. The LeakyReLU function is used to extract features of the input data and find key features related to the output results. The second and third hidden sublayers each have 128 nodes. The Sigmoid function is used to further explore the features of the output data of the first hidden sublayer and capture the intrinsic connections between nodes. The fourth and fifth hidden sublayers each contain 64 nodes. The ReLU activation function is used to perform secondary processing on the data features output by the third hidden sublayer. The output layer is provided with an output node, and the output layer adopts a linear activation function to output the final prediction result.

[0012] Furthermore, the independent neural network model is compared with the corresponding first neural network model, and the composite neural network model is compared with the corresponding second neural network model, and the correlation between the output scores under different inputs is calculated, and the substitution relationship between the models is determined based on the correlation.

[0013] A second aspect of an embodiment of the present invention provides a facial recognition system based on fatigue driving, for implementing the facial recognition method based on fatigue driving described in the first aspect, the system comprising: An acquisition module, configured to acquire historical facial images of a user and identify facial features based on the historical facial images, wherein the facial features include at least eye features, eyebrow features, and mouth features; A first establishment module is used to establish independent neural network models of eye features, eyebrow features, and mouth features, and a composite neural network model with a combination relationship, and train the independent neural network models and the composite neural network model with corresponding historical facial features; An input module is used to obtain a facial image of the user in real time during vehicle driving, determine unobstructed target features, input the target features into the corresponding trained independent neural network model or the composite neural network model, and output a fatigue driving score; The output model is used to output the fatigue driving detection result according to the fatigue driving score.

[0014] A third aspect of an embodiment of the present invention provides a computer-readable storage medium having a computer program stored thereon. When the program is executed by a processor, the facial recognition method based on fatigue driving provided in the first aspect is implemented.

[0015] A fourth aspect of an embodiment of the present invention provides an electronic device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein when the processor executes the program, the facial recognition method based on fatigue driving provided in the first aspect is implemented.

[0016] A facial recognition method and system based on fatigue driving provided in an embodiment of the present invention obtains a user's historical facial image, identifies facial features based on the historical facial image, and the facial features include at least eye features, eyebrow features and mouth features; establishes independent neural network models of eye features, eyebrow features and mouth features and a composite neural network model with a combination relationship, and trains the independent neural network model and the composite neural network model with the corresponding historical facial features; during vehicle driving, obtains the user's facial image in real time, determines the unobstructed target features, and inputs the target features into the corresponding trained independent neural network model or composite neural network model, and outputs a fatigue driving score; outputs a fatigue driving detection result based on the fatigue driving score. Specifically, through the above method, fatigue driving detection can still be performed when there are obstructions on the face, and the sensitivity of the detection is guaranteed. BRIEF DESCRIPTION OF THE DRAWINGS

[0017] Figure 1 A flowchart of a facial recognition method based on fatigue driving provided in the first embodiment of the present invention; Figure 2 This is a structural block diagram of a facial recognition system based on fatigue driving provided in the third embodiment of the present invention; Figure 3 This is a structural block diagram of an electronic device provided in Example 4 of the present invention. DETAILED DESCRIPTION

[0018] To facilitate understanding of the present invention, the present invention will be described more fully below with reference to the accompanying drawings. The drawings illustrate several embodiments of the present invention. However, the present invention may be implemented in many different forms and is not limited to the embodiments described herein. Rather, these embodiments are provided to provide a more thorough and comprehensive understanding of the present invention.

[0019] It should be noted that when an element is referred to as being "fixed to" another element, it may be directly on the other element or there may be an intermediate element. When an element is referred to as being "connected to" another element, it may be directly connected to the other element or there may be an intermediate element. The terms "vertical," "horizontal," "left," "right," and similar expressions used herein are for illustrative purposes only.

[0020] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by one skilled in the art to which this invention pertains. The terms used in this specification of the present invention are for the purpose of describing specific embodiments only and are not intended to limit the present invention. The term "and / or" as used herein includes any and all combinations of one or more of the associated listed items.

[0021] Example 1 According to an embodiment of the present invention, an embodiment of a facial recognition method based on fatigue driving is provided. It should be noted that the steps shown in the flowchart of the accompanying drawings can be executed in a computer system such as a set of computer executable instructions, and although a logical order is shown in the flowchart, in some cases, the steps shown or described can be executed in an order different from that shown here.

[0022] In this embodiment, a facial recognition method based on fatigue driving is provided, which is applied in the scenario of assisted driving and can be used in electronic devices such as computers. Figure 1 , Figure 1 A flowchart of an implementation method of a facial recognition method based on fatigue driving provided in the first embodiment of the present invention is shown, which specifically includes steps S01 to S04.

[0023] Step S01: Obtain a historical facial image of the user, and identify facial features based on the historical facial image, where the facial features include at least eye features, eyebrow features, and mouth features.

[0024] Among them, first, historical facial images of the user in different fatigue states are obtained, and the various parts such as eyes, eyebrows and mouth in the historical facial images are determined, and the corresponding eye features, eyebrow features and mouth features are captured. It can be understood that eye features may include blinking frequency, eyelid closure duration, pupil diameter, pupil reaction speed, eye openness (aspect ratio calculated by eye landmarks (such as eye corners, eyelid key points)) and blink type (normal blink, long blink and incomplete blink), etc.

[0025] Eyebrow characteristics may include eyebrow position (such as the vertical height of the eyebrow head, brow peak, and eyebrow tail. Under normal circumstances, the eyebrows are naturally stretched. When tired, they may droop as a whole or unilaterally due to muscle relaxation), eyebrow spacing (the horizontal distance between the two eyebrows. When tired, the spacing may slightly increase due to distraction or decreased facial muscle tension), eyebrow movement frequency and amplitude (when awake, the eyebrows naturally move slightly with expression. When tired, the movement frequency decreases, the amplitude decreases, and the eyebrows may even remain still for a long time), and the degree of eyebrow wrinkling (under normal circumstances, the eyebrows are smooth. When tired or sleepy, they may frown unconsciously, and the contraction of the eyebrow muscles causes the wrinkles at the brow arch to deepen).

[0026] Mouth features can include mouth openness (the vertical distance between the mouth corners and the key points of the upper and lower lips is calculated. When normally closed, the distance is close to 0. When tired, yawning or muscle relaxation may cause the openness to increase. The threshold is usually set to more than 15 pixels or an actual distance of more than 3 mm), yawn frequency (the number of yawns per unit time, the frequency is low when awake, and yawns frequently when tired due to low blood oxygen. The duration of a single yawn often exceeds 3 seconds), mouth corner position (in a normal state, the mouth corners are symmetrical and in a natural horizontal position. When tired, one or both corners of the mouth may droop due to facial muscle weakness. A vertical distance difference of more than 5 pixels between the left and right corners of the mouth is considered abnormal), lip tightness (the lips are naturally relaxed when awake. When tired or sleepy, the lips may be unconsciously pursed, resulting in thinner lips and deeper lip lines), Mouth Aspect Ratio (MAR, the aspect ratio is calculated by the lip contour key points. The formula is similar to EAR. MAR>0.3 usually indicates that the mouth is open. Combined with time series, a continuous open mouth state can be determined).

[0027] Step S02: Establish independent neural network models of eye features, eyebrow features, and mouth features, and a composite neural network model with a combination relationship, and train the independent neural network models and the composite neural network model with corresponding historical facial features.

[0028] Since users may wear sunshades, such as sunglasses and masks, while driving, not all facial features can be recognized. Therefore, multiple different neural network models are trained to adapt to the corresponding situations to avoid inputting all available features into one neural network model, which leads to large computational complexity and inaccurate output results.

[0029] Specifically, establishing an independent neural network model for eye features refers to a neural network model whose input is only eye features; establishing an independent neural network model for eyebrow features refers to a neural network model whose input is only eyebrow features; establishing an independent neural network model for mouth features refers to a neural network model whose input is only mouth features; the composite neural network model includes a neural network model whose input is eye features and eyebrow features, a neural network model whose input is eye features and mouth features, a neural network model whose input is eyebrow features and mouth features, and a neural network model whose input is eye features, eyebrow features, and mouth features.

[0030] The input of the neural network model is facial features, and the output is the score of fatigue driving, which can be given according to expert evaluation.

[0031] Specifically, the neural network model consists of an input layer, a hidden layer, and an output layer. The input layer includes a number of input nodes, the number of which is the same as the number of input data types, and is used to receive different types of input data. The hidden layer consists of five sublayers. The first hidden sublayer has 64 nodes. The LeakyReLU function is used to extract features of the input data and find key features related to the output results. The second and third hidden sublayers each have 128 nodes. The Sigmoid function is used to further explore the features of the output data of the first hidden sublayer and capture the intrinsic connections between nodes. The fourth and fifth hidden sublayers each contain 64 nodes. The ReLU activation function is used to perform secondary processing on the data features output by the third hidden sublayer. The output layer is provided with an output node, and the output layer adopts a linear activation function to output the final prediction result.

[0032] Specifically, 80% of the randomly shuffled data set samples are used as training set samples, and the remaining 20% ​​are used as test set samples. The neuron formulas of the hidden layer and the output layer can be expressed as: First hidden sublayer: ; Second hidden sublayer: ; The third hidden sublayer: ; Fourth hidden sublayer: ; Fifth hidden sublayer: ; Output layer: ; Among them, x j and y i represent the jth node of the input layer and the ith node of the output layer respectively, Indicates the connection The weight of the j-th node in the input layer and the i-th node in the output layer, Indicates the The bias term of the i-th node in the layer, for example, is represented as the i-th neuron in the first hidden sublayer, is represented as the i-th neuron in the second hidden sublayer, is represented as the i-th neuron in the third hidden sublayer, is represented as the i-th neuron in the fourth hidden sublayer, Represented as the i-th neuron in the fifth hidden sublayer. Similarly, is represented as the jth neuron in the first hidden sublayer, is represented as the jth neuron in the second hidden sublayer, is represented as the jth neuron in the third hidden sublayer, is represented as the jth neuron in the fourth hidden sublayer, It represents the jth neuron in the fifth hidden sublayer. LeakyReLU, Sigmoid, and ReLU are activation functions.

[0033] Step S03: During vehicle driving, the user's facial image is acquired in real time, unobstructed target features are determined, and the target features are input into the corresponding trained independent neural network model or the composite neural network model to output a fatigue driving score.

[0034] It should be noted that the user's facial image is obtained in real time to determine the unobstructed target features. When the target feature is unique, it is input into the corresponding trained independent neural network model. For example, when the user wears a mask and only the eye feature is recognized, the eye feature is input into the corresponding independent neural network model and the score is output. When the target feature is not unique, it is input into the corresponding trained composite neural network model. For example, when the user wears sunglasses and the eyebrow feature and mouth feature can be recognized, the eyebrow feature and mouth feature are simultaneously input into the corresponding independent neural network model and the score is output.

[0035] Step S04: outputting the fatigue driving detection result according to the fatigue driving score.

[0036] When the score exceeds a preset value, it is determined that the user is in a fatigue driving state and an alarm is issued.

[0037] In summary, the facial recognition method based on fatigue driving in the above embodiments of the present invention obtains the user's historical facial images, and identifies facial features based on the historical facial images, and the facial features include at least eye features, eyebrow features and mouth features; establishes independent neural network models of eye features, eyebrow features and mouth features and a composite neural network model with a combination relationship, and trains the independent neural network model and the composite neural network model with the corresponding historical facial features; during the vehicle driving process, the user's facial image is obtained in real time, the unobstructed target features are determined, and the target features are input into the corresponding trained independent neural network model or composite neural network model, and the fatigue driving score is output; according to the fatigue driving score, the fatigue driving detection result is output. Specifically, through the above method, fatigue driving detection can still be performed when there are obstructions on the face, and the detection sensitivity is guaranteed.

[0038] Example 2 The second embodiment of the present invention also provides a facial recognition method based on fatigue driving. The difference from the first embodiment of the present invention is that, in order to improve the accuracy of fatigue driving detection, additional features are introduced for model training. Specifically: According to the target features and the number of target features, a neural network model with different inputs and the same output is established and trained. The fewer the number of target features, the more types of inputs the neural network model can have.

[0039] It should be noted that since eye features are more critical, most fatigue driving can be detected through eye features, but there may be certain inaccuracies when only using eyebrow features and mouth features. Therefore, relevant features of the driving process are introduced to train a new neural network model for fatigue driving detection.

[0040] In the embodiment of the present invention, a first neural network model of the eyebrow feature, the first feature and the second feature, and the mouth feature, the first feature and the second feature is established; Establishing a second neural network model of eyebrow features, mouth features and the first feature; The first feature is either an in-car atmosphere feature or a driving feature. When the first feature is an in-car atmosphere feature, the second feature is the driving feature. When the first feature is the driving feature, the second feature is the in-car atmosphere feature. The in-car atmosphere feature is used to characterize the sleeping environment in the car, and the driving feature characterizes the user's driving conditions. It is understandable that when the input feature is only one of the eyebrow feature or the mouth feature, the additional first and second features need to be added to train the first neural network model. When the input features include both the eyebrow feature and the mouth feature, one of the first and second features is introduced to train the second neural network model.

[0041] Specifically, the in-car atmosphere characteristics include at least in-car music characteristics, communication characteristics, and temperature characteristics. The in-car music characteristics include at least volume sub-features and melody sub-features. The communication characteristics include at least decibel sub-features and emotion sub-features.

[0042] In-car music feature extraction, the volume sub-feature is obtained through time-domain analysis of the audio signal. This involves real-time acquisition of audio sample amplitude data, calculation of short-term average energy (reflecting volume), root mean square (RMS, a measure of sound intensity), peak volume (maximum amplitude in the audio clip), and dynamic range (the difference between peak and minimum values), while also counting volume mutations (e.g., the frequency of sudden silences or sudden increases in volume). The melody sub-feature relies on frequency-domain analysis and music information retrieval (MIR) techniques: Fourier transforms are used to convert audio signals into frequency-domain data. Pitch detection algorithms (such as YIN and pYIN) are then used to extract the pitch sequence of the melody line. The interval size (number of semitones) between adjacent notes, pitch change direction (ascending / falling / sustained), melodic contour entropy (a measure of the complexity of pitch fluctuations), and modal center (determining the tonic through chromaticity eigenvectors) are then calculated. The synchronization between melody and rhythm is then analyzed using beat detection results. The two types of features can be automatically extracted through audio processing libraries such as librosa and Essentia, providing an acoustic basis for analyzing the driver's emotional state in fatigue driving detection.

[0043] In communication feature extraction, the decibel sub-feature can collect the time domain amplitude data of the voice signal through the microphone, and use the acoustic measurement algorithm to calculate the real-time sound pressure level (SPL), usually in decibels (dB). The microphone must first be calibrated to eliminate environmental noise interference, and then the average decibel value, peak decibel value and decibel fluctuation range during speaking are obtained through short-time energy analysis (for example, normal conversation is about 40-60dB, and may exceed 70dB when emotional). Emotional sub-features rely on natural language processing (NLP) and speech emotion recognition (SER) technologies. These features perform endpoint detection on speech signals to isolate valid speech segments. Acoustic parameters (such as pitch, formants, speaking rate, and pause duration) are extracted using features such as MFCC (Mel-Frequency Cepstral Coefficients) and spectrograms. Pre-trained deep learning models (such as LSTM and Transformer) are then used to analyze emotional tendencies in speech (e.g., calm, anxious, angry, excited). Furthermore, the communication content is converted to text, and sentiment polarity scores (positive / negative / neutral) are calculated using sentiment lexicons (such as HowNet) and models like BERT. The frequency of emotional keywords (e.g., "irritated" and "tired") is also counted. These two types of features can be used to collect speech data in real time using libraries such as PyAudio and SpeechRecognition. Lightweight models can be deployed using OpenVINO or TensorFlowLite to implement on-device emotion analysis, providing multimodal data support for assessing the driver's mental state during fatigue driving detection.

[0044] The driving characteristics described herein include at least vehicle speed sub-features, steering wheel operation sub-features, and accelerator / brake pedal operation sub-features. It is understood that during vehicle driving data collection, the vehicle speed sub-feature can directly read the real-time vehicle speed signal (units: km / h or m / s) transmitted from the OBD interface via the vehicle's CAN bus, or utilize positioning data from a GPS module to calculate instantaneous speed by using the displacement difference between adjacent moments. Parameters such as speed fluctuation (the magnitude of vehicle speed change per unit time), maximum / minimum speed, and duration of speed overruns are also extracted. The steering wheel operation sub-feature uses a steering wheel angle sensor (such as a magnetoresistive or photoelectric sensor) to collect real-time steering angle (absolute value and left and right steering amplitude), steering angular velocity (steering speed), and steering duration (the duration of a single steering action). It also calculates steering wheel torque (using a torque sensor to monitor the steering force applied by the driver), self-centering speed (the rate at which the steering wheel automatically returns to center after releasing the driver's hand), and steering entropy (an indicator of steering irregularity, reflecting operational disturbances caused by driver fatigue). The accelerator / brake pedal operation sub-features rely on pedal travel sensors (such as potentiometers or Hall-effect sensors) to capture throttle opening (0%-100%), brake pedal travel (depression depth), pedal actuation frequency (number of pedal presses / releases per unit time), single pedaling duration, accelerator-brake switching delay (the time interval between accelerator and brake application), and the number of sudden acceleration / braking events (determined by the pedal travel change rate exceeding a threshold). These features are collected in real time by the onboard ECU (electronic control unit) or an external OBD device, filtered and noise-reduced, and transmitted to the fatigue detection system. Combined with time series analysis of driving behavior patterns, they provide a dynamic basis for fatigue status identification.

[0045] In some other embodiments of the present invention, to evaluate the necessity of introducing additional features, that is, to determine whether the introduction of additional features affects the output results, the independent neural network model is compared with the corresponding first neural network model, and the composite neural network model is compared with the corresponding second neural network model. The correlation between the output scores under different inputs is calculated, and the replacement relationship between the models is determined based on the correlation. If replacement is possible, the computational complexity of the model is reduced while ensuring the same results to a certain extent.

[0046] It should be noted that the correlation calculation formula is: ; in, , are the scores of the corresponding model outputs in the same period, , is the mean, n is the sample size, it is understandable, The closer the absolute value is to 1, the stronger the linear correlation is; if it is close to 0, there is no significant linear relationship. In the embodiment of the present invention, if r is greater than 0.98, it means that the outputs of the two models are highly positively correlated and have interchangeability.

[0047] Example 3 See also Figure 2 , Figure 2 This is a block diagram of a facial recognition system for fatigue driving provided in Example 3 of the present invention. The facial recognition system 200 for fatigue driving is used to implement the above-mentioned embodiments and preferred implementations, and details that have already been described will not be repeated. As used below, the term "module" may refer to a combination of software and / or hardware that implements a predetermined function. Although the devices described in the following embodiments are preferably implemented in software, implementation in hardware, or a combination of software and hardware, is also possible and contemplated.

[0048] Specifically, the facial recognition system 200 based on fatigue driving includes: a first acquisition module 21, a first establishment module 22, an input module 23 and an output module 24, wherein: An acquisition module 21 is configured to acquire a user's historical facial image and identify facial features based on the historical facial image, wherein the facial features include at least eye features, eyebrow features, and mouth features; A first establishing module 22 is used to establish independent neural network models for eye features, eyebrow features, and mouth features, and a composite neural network model having a combination relationship, and train the independent neural network models and the composite neural network model using the corresponding historical facial features. The neural network model is composed of an input layer, a hidden layer, and an output layer. The input layer includes a plurality of input nodes, the number of which is the same as the number of input data types, and is used to receive different types of input data; The hidden layer consists of five sublayers. The first hidden sublayer has 64 nodes. The LeakyReLU function is used to extract features of the input data and find key features related to the output results. The second and third hidden sublayers each have 128 nodes. The Sigmoid function is used to further explore the features of the output data of the first hidden sublayer and capture the intrinsic connections between nodes. The fourth and fifth hidden sublayers each contain 64 nodes. The ReLU activation function is used to perform secondary processing on the data features output by the third hidden sublayer. The output layer is provided with an output node, and the output layer adopts a linear activation function to output the final prediction result; An input module 23 is configured to acquire a facial image of the user in real time during vehicle driving, determine unobstructed target features, input the target features into the corresponding trained independent neural network model or the composite neural network model, and output a fatigue driving score; The output model 24 is used to output the fatigue driving detection result according to the fatigue driving score.

[0049] Furthermore, in some optional embodiments of the present invention, the facial recognition system 200 based on fatigue driving further includes: A second establishment module is used to establish and train a neural network model with different inputs and the same output according to the target features and the number of target features, wherein the fewer the number of target features, the more types of inputs the neural network model has, and to establish a first neural network model with an eyebrow feature, a first feature, and a second feature, as well as a mouth feature, a first feature, and a second feature; Establishing a second neural network model of eyebrow features, mouth features and the first feature; The first feature is an in-car atmosphere feature or a driving feature. When the first feature is the in-car atmosphere feature, the second feature is the driving feature. When the first feature is the driving feature, the second feature is the in-car atmosphere feature. The in-car atmosphere feature is used to characterize the sleeping environment in the car. The driving feature characterizes the user's driving situation. The in-car atmosphere feature includes at least an in-car music feature, a communication feature, and a temperature feature. The in-car music feature includes at least a volume sub-feature and a melody sub-feature. The communication feature includes at least a decibel sub-feature and an emotion sub-feature. The driving feature includes at least a vehicle speed sub-feature, a steering wheel operation sub-feature, and an accelerator / brake pedal operation sub-feature. In addition, the independent neural network model is compared with the corresponding first neural network model, and the composite neural network model is compared with the corresponding second neural network model, and the correlation between the output scores under different inputs is calculated, and the substitution relationship between the models is determined based on the correlation.

[0050] Example 4 Another aspect of the present invention provides an electronic device, see Figure 3 , shown is an electronic device in embodiment 4 of the present invention, including a memory 20, a processor 10, and a computer program 30 stored in the memory and executable on the processor. When the processor 10 executes the computer program 30, the facial recognition method based on fatigue driving as described above is implemented.

[0051] In some embodiments, the processor 10 may be a central processing unit (CPU), a controller, a microcontroller, a microprocessor, or other data processing chip, used to run program codes or process data stored in the memory 20, such as executing access restriction programs.

[0052] The memory 20 includes at least one type of readable storage medium, including flash memory, a hard disk, a multimedia card, a card-type memory (e.g., SD or DX memory), a magnetic memory, a magnetic disk, an optical disk, etc. In some embodiments, the memory 20 may be an internal storage unit of the electronic device, such as the hard disk of the electronic device. In other embodiments, the memory 20 may also be an external storage device of the electronic device, such as a plug-in hard disk equipped on the electronic device, a Smart Media Card (SMC), a Secure Digital (SD) card, a Flash Card, etc. Furthermore, the memory 20 may include both an internal storage unit of the electronic device and an external storage device. The memory 20 can be used not only to store application software and various types of data of the electronic device, but also to temporarily store data that has been output or is about to be output.

[0053] It should be pointed out that Figure 3 The structure shown does not constitute a limitation to the electronic device. In other embodiments, the electronic device may include fewer or more components than shown in the figure, or combine certain components, or arrange the components differently.

[0054] An embodiment of the present invention further provides a computer-readable storage medium having a computer program stored thereon. When the program is executed by a processor, the facial recognition method based on fatigue driving as described above is implemented.

[0055] Those skilled in the art will appreciate that the logic and / or steps represented in the flowcharts or otherwise described herein, for example, can be considered as a sequenced list of executable instructions for implementing logical functions, and can be embodied in any computer-readable medium for use by an instruction execution system, apparatus, or device (e.g., a computer-based system, a system including a processor, or other system that can fetch and execute instructions from an instruction execution system, apparatus, or device), or in conjunction with such instruction execution system, apparatus, or device. For purposes of this specification, "computer-readable medium" can be any device that can contain, store, communicate, propagate, or transport a program for use by an instruction execution system, apparatus, or device, or in conjunction with such instruction execution system, apparatus, or device.

[0056] More specific examples (a non-exhaustive list) of computer-readable media include the following: an electrical connection with one or more wires (electronic devices), a portable computer disk cartridge (magnetic devices), a random access memory (RAM), a read-only memory (ROM), an erasable and programmable read-only memory (EPROM or flash memory), a fiber optic device, and a portable compact disc read-only memory (CDROM). In addition, the computer-readable medium may even be paper or other suitable medium on which the program is printed, since the program may be obtained electronically, for example, by optically scanning the paper or other medium and then editing, interpreting, or processing it in another suitable manner as necessary, and then storing it in a computer memory.

[0057] It should be understood that various components of the present invention may be implemented using hardware, software, firmware, or a combination thereof. In the aforementioned embodiments, multiple steps or methods may be implemented using software or firmware stored in a memory and executed by a suitable instruction execution system. For example, if implemented using hardware, as in another embodiment, any one or a combination of the following technologies known in the art may be used: a discrete logic circuit having logic gate circuits for implementing logic functions on data signals, an application-specific integrated circuit having suitable combinational logic gate circuits, a programmable gate array (PGA), a field-programmable gate array (FPGA), etc.

[0058] Throughout this specification, reference to terms such as "one embodiment," "some embodiments," "examples," "specific examples," or "some examples" means that a specific feature, structure, material, or characteristic described in conjunction with that embodiment or example is included in at least one embodiment or example of the present invention. In this specification, schematic representations of the above terms do not necessarily refer to the same embodiment or example. Furthermore, the specific features, structures, materials, or characteristics described may be combined in any suitable manner in any one or more embodiments or examples.

[0059] The above embodiments merely illustrate several implementations of the present invention, and while their descriptions are relatively specific and detailed, they should not be construed as limiting the scope of the present invention. It should be noted that a person skilled in the art would be able to make numerous modifications and improvements without departing from the spirit of the present invention, all of which fall within the scope of protection of the present invention. Therefore, the scope of protection of the present invention shall be determined by the appended claims.

Claims

1. A facial recognition method based on fatigue driving, characterized in that: The method comprises: Obtaining a historical facial image of the user, and identifying facial features based on the historical facial image, the facial features including at least eye features, eyebrow features, and mouth features; Establishing independent neural network models for eye features, eyebrow features, and mouth features, and a composite neural network model with a combination relationship, and training the independent neural network models and the composite neural network model using the corresponding historical facial features; During vehicle driving, a facial image of the user is acquired in real time, unobstructed target features are determined, and the target features are input into the corresponding trained independent neural network model or the composite neural network model to output a fatigue driving score; According to the fatigue driving score, the fatigue driving detection result is output.

2. The facial recognition method based on fatigue driving according to claim 1, characterized in that: The method further comprises: According to the target features and the number of target features, a neural network model with different inputs and the same output is established and trained. The fewer the number of target features, the more types of inputs the neural network model can have.

3. The facial recognition method based on fatigue driving according to claim 2, characterized in that: In the step of establishing and training a neural network model with different inputs and the same output according to the target features and the number of target features, Establishing a first neural network model of the eyebrow feature, the first feature, and the second feature, as well as the mouth feature, the first feature, and the second feature; Establishing a second neural network model of eyebrow features, mouth features and the first feature; Among them, the first feature is the in-car atmosphere feature or the driving feature. When the first feature is the in-car atmosphere feature, the second feature is the driving feature. When the first feature is the driving feature, the second feature is the in-car atmosphere feature. The in-car atmosphere feature is used to characterize the sleeping environment in the car, and the driving feature characterizes the user's driving condition.

4. The facial recognition method based on fatigue driving according to claim 3, characterized in that: The in-car atmosphere features include at least in-car music features, communication features, and temperature features. The in-car music features include at least volume sub-features and melody sub-features. The communication features include at least decibel sub-features and emotion sub-features.

5. The facial recognition method based on fatigue driving according to claim 4, characterized in that: The driving characteristics include at least vehicle speed sub-characteristics, steering wheel operation sub-characteristics, and accelerator / brake pedal operation sub-characteristics.

6. The facial recognition method based on fatigue driving according to claim 5, characterized in that: The neural network model consists of an input layer, a hidden layer, and an output layer. The input layer contains a number of input nodes, the number of which is the same as the number of input data types, and is used to receive different types of input data. The hidden layer consists of five sublayers. The first hidden sublayer has 64 nodes. The LeakyReLU function is used to extract features of the input data and find key features related to the output results. The second and third hidden sublayers each have 128 nodes. The Sigmoid function is used to further explore the features of the output data of the first hidden sublayer and capture the intrinsic connections between nodes. The fourth and fifth hidden sublayers each contain 64 nodes. The ReLU activation function is used to perform secondary processing on the data features output by the third hidden sublayer. The output layer is provided with an output node, and the output layer adopts a linear activation function to output the final prediction result.

7. The facial recognition method based on fatigue driving according to claim 6, characterized in that: Compare the independent neural network model with the corresponding first neural network model, and compare the composite neural network model with the corresponding second neural network model, calculate the correlation between the output scores under different inputs, and determine the substitution relationship between the models based on the correlation.

8. A facial recognition system based on fatigue driving, characterized in that: For implementing the facial recognition method based on fatigue driving according to any one of claims 1 to 7, the system comprises: An acquisition module, configured to acquire historical facial images of a user and identify facial features based on the historical facial images, wherein the facial features include at least eye features, eyebrow features, and mouth features; A first establishment module is used to establish independent neural network models of eye features, eyebrow features, and mouth features, and a composite neural network model with a combination relationship, and train the independent neural network models and the composite neural network model with corresponding historical facial features; An input module is used to obtain a facial image of the user in real time during vehicle driving, determine unobstructed target features, input the target features into the corresponding trained independent neural network model or the composite neural network model, and output a fatigue driving score; The output model is used to output the fatigue driving detection result according to the fatigue driving score.

9. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the program is executed by a processor, the facial recognition method based on fatigue driving as described in any one of claims 1 to 7 is implemented.

10. An electronic device, characterized in that: The invention comprises a memory, a processor and a computer program stored in the memory and executable on the processor, wherein when the processor executes the program, the facial recognition method based on fatigue driving according to any one of claims 1 to 7 is implemented.

Images