Two-dimensional holder face positioning and fatigue driving detection device

Abstract

The invention relates to the technical field of driving behavior safety recognition, in particular to a two-dimensional holder face positioning and fatigue driving detection device which comprises a data collection and storage module, an image information recognition and processing module, a face analysis and fatigue driving evaluation module and an intelligent alarm module. A two-dimensional image lossless storage mechanism is established in the data collection and storage module, the face image information is compressed and stored, and a two-dimensional image storage information base is obtained; a two-dimensional image feature recognition model and a two-dimensional image processing model are arranged in the image information recognition and processing module, a two-dimensional image information base is recognized and processed based on the models, and face monitoring information is obtained; a face analysis model and a fatigue driving evaluation system are established in the face analysis and fatigue driving evaluation module, and the module receives and processes the face monitoring information and outputs a fatigue driving evaluation result; and the intelligent alarm module sends a warning signal according to the evaluation result, so that the fatigue driving state is monitored and alarmed.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of driving behavior safety recognition, in particular to a two-dimensional holder face positioning and fatigue driving detection device. BACKGROUND

[0002] Real-time monitoring of fatigue driving is a major focus in the field of automotive safety. In practical applications, as the monitoring time continues to extend and the monitoring range continues to expand, the accumulated face image data and driving monitoring data of the device and system show a growth trend. However, the current face positioning and fatigue driving detection device is difficult to cope with the processing and management needs of massive databases in data storage technology, which not only causes the risk of loss or damage of device data, but also may cause adverse effects on subsequent fatigue driving detection analysis and practical application.

[0003] Although the face positioning technology has made certain progress, in complex and variable driving scenarios, such as rapid changes in light conditions, drivers wearing glasses or hats and other obstructions, its accuracy and stability still face severe challenges. At the same time, the monitoring process of fatigue driving is also restricted by factors such as individual differences of drivers and driving habits, which makes it difficult for the existing device to meet the real-time demand in monitoring accuracy, and may cause information misreporting or missing report, etc., which poses a potential threat to driving safety.

[0004] Therefore, it is urgent to set up a more intelligent driving detection data storage, processing and analysis device to ensure the integrity and availability of relevant information during driving, which not only helps to improve the feasibility and practicality of face information storage, but also provides strong support for effective analysis and identification of face features and key information of fatigue driving, and further provides a more solid guarantee for driving safety. SUMMARY

[0005] In view of the deficiencies of the existing method and the requirements of practical application, in order to adapt to the data storage requirements in practical application, guarantee the effective analysis and identification of face features and fatigue driving key information, the data storage technology is optimized to guarantee the stability and accuracy of the monitoring database in complex environment, and to provide strong support for the effective analysis and accurate identification of face features and fatigue driving key information, and then guarantee the effective implementation and application of face positioning and fatigue driving detection technology. On the one hand, the present application provides a two-dimensional holder face positioning and fatigue driving detection device, which comprises: a data collection and storage module, an image information identification and processing module, a face analysis and fatigue driving evaluation module and an intelligent alarm module; a two-dimensional image lossless storage mechanism is established in the data collection and storage module, the face image information in the driving process is compressed and stored according to the two-dimensional image lossless storage mechanism, and a two-dimensional image storage information base in the driving process is obtained; a two-dimensional image feature identification model and a two-dimensional image processing model are arranged in the image information identification and processing module, the two-dimensional image storage information base is identified and processed based on the two-dimensional image feature identification model and the two-dimensional image processing model, so as to obtain the face monitoring information in the driving process; a face analysis model and a fatigue driving evaluation system are established in the face analysis and fatigue driving evaluation module, the face analysis and fatigue driving evaluation module receives and processes the face monitoring information, and outputs the fatigue driving evaluation result in the driving process; the intelligent alarm module sends fatigue warning signal in the driving process according to the fatigue driving evaluation result, and realizes real-time monitoring and early warning of fatigue driving state based on the fatigue warning signal. The device of the present application adopts a modular design method, which is easy to expand and upgrade, and introduces advanced data storage technology, identification algorithm and face analysis model, which can adapt to the changing data storage requirements and driving safety monitoring targets.

[0006] Optionally, the two-dimensional image lossless storage mechanism established in the data collection and storage module comprises: obtaining face image information in the driving process through the data collection and storage module; a two-dimensional image compression storage model is established in the two-dimensional image lossless storage mechanism based on the information requirements of the fatigue driving detection device and the face image information. The two-dimensional image compression storage model is established, so that the device can effectively compress the face image information, thereby reducing the requirement of data storage space, and the device can save as much image data as possible, which is particularly important for long-time and continuous driving monitoring.

[0007] Optionally, the face image information in the driving process is compressed and stored according to the two-dimensional image lossless storage mechanism, and a two-dimensional image storage information library in the driving process is obtained, and the two-dimensional image lossless storage mechanism compresses and stores the face image information through the two-dimensional image compression storage model to obtain the two-dimensional image storage information library in the driving process. The two-dimensional image compression storage model satisfies the following relationship: wherein, represents the output result of the information encoder in the nth iteration process, represents a first function corresponding to the ConvLSTM recurrent network, represents a second function corresponding to the ConvLSTM recurrent network, represents a third function corresponding to the ConvLSTM recurrent network, represents a corresponding neuron weight coefficient, represents a corresponding neuron weight coefficient, represents a corresponding neuron weight coefficient, represents a normalized factor of encoding compression, represents a convolutional neural network function, represents an initial two-dimensional image of the driver in the driving process, represents an influence weight of the accumulated neural network function. The two-dimensional image compression storage model of the application compresses and stores the face image information in the driving process, not only improves the information storage efficiency and quality, but also enhances the adaptability and intelligent level of the device, and provides a solid foundation for the wide application of the fatigue driving detection device.

[0008] Optionally, the two-dimensional image feature recognition model and the two-dimensional image processing model are arranged in the image information recognition and processing module, and the two-dimensional image feature recognition model is established based on a binary recurrent network structure. The two-dimensional image feature recognition model satisfies the following relationship: wherein, represents a binary mapping result of the two-dimensional compressed image feature data, represents an auxiliary constant in the two-dimensional image feature recognition process, represents a binary mapping function, represents the output result of the information encoder in the nth iteration process, represents a neuron weight coefficient in a single-layer ConvLSTM recurrent network structure, represents a numerical adjustment item representing a binary mapping. The application establishes a two-dimensional image feature recognition model based on a binary cyclic network structure, which not only improves the efficiency and accuracy of feature extraction, but also reduces the computational complexity, enhances the adaptability of the model, promotes the intelligent development of the device, and reduces the false positive and false negative rates of data.

[0009] Optionally, the two-dimensional image feature recognition model and the two-dimensional image processing model are arranged in the image information recognition and processing module, including: establishing a two-dimensional image processing model based on a decoder neural network structure; The two-dimensional image processing model satisfies the following relationship: Among them, represents the recovered two-dimensional image data, represents the corresponding weight coefficient, represents the first neuron function of the convolutional neural network, represents the information set after the image feature binary mapping, represents the auxiliary parameter of the depth compression layer, represents the first neuron function of the cyclic network in the processing model, represents the second neuron function of the cyclic network in the processing model, represents the third neuron function of the cyclic network in the processing model, represents the fourth neuron function of the cyclic network in the processing model, represents the corresponding weight coefficient, represents the corresponding weight coefficient, represents the corresponding weight coefficient, represents the corresponding weight coefficient, represents the corresponding weight coefficient, represents the second neuron function of the convolutional neural network. The two-dimensional image processing model established by the application not only improves the quality of image restoration and the accuracy of feature information, but also helps to improve the practical performance of the fatigue driving detection device.

[0010] Optionally, the identifying and processing the two-dimensional image storage information base based on the two-dimensional image feature recognition model and the two-dimensional image processing model to obtain the face monitoring information in the driving process comprises: identifying the two-dimensional image storage information base by using the two-dimensional image feature recognition model to obtain a feature mapping result of a two-dimensional compressed image; processing the feature mapping result by using the two-dimensional image processing model to obtain two-dimensional feature image restoration information. The face monitoring information in the driving process is obtained based on the feature mapping result and the two-dimensional feature image restoration information. The two-dimensional image feature recognition model can accurately capture the key features in the image and reflect them through the feature mapping result, so that the face information of the driver can be more accurately identified, and the probability of misidentification and the like can be reduced.

[0011] Optionally, the establishing the face analysis model and the fatigue driving evaluation system in the face analysis and fatigue driving evaluation module comprises: establishing the face analysis model based on the fatigue driving detection requirement, wherein the face analysis model comprises a face recognition algorithm, an eye recognition algorithm, a two-dimensional pan-tilt control algorithm, a PERCLOS algorithm and a blink frequency algorithm. The face recognition algorithm can recognize the facial features of the driver, the eye recognition algorithm focuses on the eye features of the driver, and the PERCLOS algorithm and the blink frequency algorithm are important indicators for evaluating fatigue driving, so that the device can accurately judge the fatigue degree of the driver.

[0012] Optionally, the establishing the face analysis model and the fatigue driving evaluation system in the face analysis and fatigue driving evaluation module comprises: establishing the fatigue driving evaluation system based on the analysis result of the face analysis model and the two-dimensional image storage information base, wherein the fatigue driving evaluation system comprises a face geometric center condition, an eye closing state, a blink frequency, a head posture stability, a facial expression recognition and a driving time and rest time. The fatigue driving evaluation system can monitor the facial features of the driver in real time, so that the device can quickly respond to the fatigue state of the driver and timely issue a warning signal.

[0013] Optionally, the face analysis and fatigue driving evaluation module receives and processes the face monitoring information, and outputs the fatigue driving evaluation result in the driving process, comprising: the face analysis and fatigue driving evaluation module obtains the face analysis result in the driving process through the face recognition algorithm, the human eye recognition algorithm, the two-dimensional pan-tilt control algorithm, the PERCLOS algorithm and the blink frequency algorithm; the face analysis and fatigue driving evaluation module outputs the fatigue driving evaluation result in the driving process in combination with the face analysis result and the fatigue driving evaluation system. The face analysis and fatigue driving evaluation module of the present application realizes real-time monitoring and analysis of the facial features of the driver and outputs the fatigue driving evaluation result through a series of algorithms and evaluation systems, which helps to improve driving safety, enhance the intelligent level of the device, and promote technology development and application.

[0014] Optionally, the intelligent alarm module sends a fatigue warning signal in the driving process according to the fatigue driving evaluation result, and realizes real-time monitoring and early warning of the fatigue driving state based on the fatigue warning signal, comprising: configuring a safety warning device and a remote communication device in the alarm module; in combination with the safety warning device, the remote communication device, the fatigue driving evaluation result, the fatigue driving state in the driving process is detected and intelligently warned in real time, so as to realize real-time monitoring and early warning of the fatigue driving state. The present application combines a variety of algorithms and evaluation systems, wherein the intelligent alarm module can accurately judge the fatigue degree of the driver and send corresponding warning signals, which helps to reduce the possibility of false and missed reports, and improves the practicability of the face positioning and fatigue driving detection device. BRIEF DESCRIPTION OF DRAWINGS

[0015] Figure 1 Flow chart of the two-dimensional pan-tilt face positioning and fatigue driving detection device of the present application; Figure 2 Physical assembly schematic diagram of the two-dimensional pan-tilt in the two-dimensional pan-tilt face positioning and fatigue driving detection device of the present application; Figure 3 Motion direction schematic diagram of the two-dimensional pan-tilt in the two-dimensional pan-tilt face positioning and fatigue driving detection device of the present application; Figure 4 Schematic diagram of the principle of the blink frequency algorithm in the two-dimensional pan-tilt face positioning and fatigue driving detection device of the present application; Figure 5 Structural diagram of the two-dimensional pan-tilt face positioning and fatigue driving detection device of the present application. DETAILED DESCRIPTION

[0016] Specific embodiments of the present application will now be described in detail with reference to the drawings, which are provided by way of example only and are not intended to limit the present application. In the following description, numerous specific details are set forth to provide a thorough understanding of the present application. However, it will be apparent to one of ordinary skill in the art that the present application can be practiced without requiring these specific details. In other instances, well-known circuits, software or methods have not been described in detail in order to avoid obscuring the present application.

[0017] Reference throughout this specification to "one embodiment", "an embodiment", "one example", or "an example" means that a particular feature, structure, or characteristic described in connection with the embodiment or example is included in at least one embodiment of the application. Thus, appearances of the phrases "in one embodiment" or "in an embodiment" or "one example" or "an example" in various places throughout this specification are not necessarily all referring to the same embodiment or example. Furthermore, the particular features, structures, or characteristics can be combined in any suitable

[0018] Reference throughout this specification to "one embodiment", "an embodiment", "one example", or "an example" means that a particular feature, structure, or characteristic described in connection with the embodiment or example is included in at least one embodiment of the application. Thus, appearances of the phrases "in one embodiment" or "in an embodiment" or "one example" or "an example" in various places throughout this specification are not necessarily all referring to the same embodiment or example. Furthermore, the particular features, structures, or characteristics can be combined in any suitable Figure 1 In order to address the data storage challenges in practical applications, the present application ensures accurate analysis and identification of key information such as facial features and fatigue driving, optimizes the data storage module of the device, enhances the stability and accuracy of the facial monitoring database in complex environments, provides a data basis for effective analysis of facial features and fatigue driving, and thus ensures the effective implementation and wide application of facial positioning and fatigue driving detection technology. The present application provides a two-dimensional gimbal facial positioning and fatigue driving detection device, which comprises the following steps: In the two-dimensional gimbal facial positioning and fatigue driving detection device, a data collection and storage module, an image information recognition and processing module, a facial analysis and fatigue driving evaluation module, and an intelligent alarm module are provided.

[0019] S1, a two-dimensional image lossless storage mechanism is established in the data collection and storage module, the facial image information during driving is compressed and stored according to the two-dimensional image lossless storage mechanism, and a two-dimensional image storage information library during driving is obtained, and the specific implementation steps and related contents are as follows: First, the facial image information during driving is obtained through the data collection and storage module.

[0020] The data collection and storage module is installed with related data acquisition equipment, which can capture the facial image information during driving in real time, and accurately record the image data during the entire driving process, further ensuring the integrity and effectiveness of the device information library.

[0021] Then, based on the information requirement of the fatigue driving detection device and the face image information, a two-dimensional image compression storage model is established in the two-dimensional image lossless storage mechanism.

[0022] In order to improve the lossless compression efficiency of the device for face image information, in the embodiment, the RNN recurrent neural network is introduced and improved in combination with the information requirement of the fatigue driving detection device and the characteristics of the face image information, and on this basis, a two-dimensional image compression storage model is established in the two-dimensional image lossless storage mechanism.

[0023] In the two-dimensional image lossless storage mechanism, the face image information output by the data collection and storage module is compressed and stored through the two-dimensional image compression storage model, so that a two-dimensional image storage information library in the driving process can be obtained.

[0024] In an optional embodiment, when the two-dimensional image lossless storage mechanism is constructed, it is divided into two core components: the encoding end (compression end) and the decoding end (decompression end), wherein the encoding end (compression end) is arranged in the data collection and storage module, and the decoding end (decompression end) is arranged in the image information recognition and processing module. In order to improve the data compression efficiency and quality of the device, the recurrent neural network is optimized by using the convolutional neural network, and an encoder network, a binary network and a decoder network are established, which further provides technical support for the storage and transmission of face image information in the driving process.

[0025] Based on the further optimization of the performance of the recurrent neural network by the convolutional neural network, the abnormal fluctuation of the gradient in the convolutional layer can be effectively inhibited, so as to ensure the accuracy of the compression and decoding of the two-dimensional image information. In the embodiment, a multi-layer neural network structure is constructed. The above multi-layer network structure can extract and encode the features of the two-dimensional image in depth, finally obtain high-quality two-dimensional image feature data compression results, and realize effective compression processing of two-dimensional image big data.

[0026] In the construction process of the two-dimensional image compression storage model, the convolutional neural network and the three-layer convolutional long short-term memory network (ConvLSTM) recurrent network are specifically fused. The convolutional neural network is configured with 64 convolutional kernels with a size of 3x3, and the ConvLSTM recurrent network is optimized and upgraded. Each layer of the network contains two kinds of regular convolutional kernels and hidden convolutional kernels. The size of the regular convolutional kernel is 3x3, and the number of different levels of the ConvLSTM recurrent network is 256 (first layer), 512 (second layer) and 512 (third layer) respectively. The size of the hidden convolutional kernel is 1x1, and the number of each layer is 64.

[0027] In the embodiment, the original two-dimensional image is set as a color image, and the corresponding size is , where H represents the height of the image, W represents the width of the image, and 3 represents the three color channels in a color image, i.e., red, green, and blue, which together constitute the color information of the image. After deep extraction and encoding of image features by the convolutional neural network in the two-dimensional image compression storage model and the three-layer ConvLSTM recurrent network, a two-dimensional image data compression result with a size of can be obtained.

[0028] The above two-dimensional image compression storage model satisfies the following relationship: wherein, represents the output result of the information encoder in the nth iteration process, represents the first function corresponding to the ConvLSTM recurrent network, represents the second function corresponding to the ConvLSTM recurrent network, represents the third function corresponding to the ConvLSTM recurrent network, represents the neuron weight coefficient corresponding to represents the neuron weight coefficient corresponding to represents the neuron weight coefficient corresponding to represents the normalization factor of encoding compression, represents the convolutional neural network function, represents the initial two-dimensional image of the driver during driving, represents the influence weight of the convolutional neural network function. The output result of the encoder network in the nth iteration process, i.e., the encoding and compression result output after multiple processing of the two-dimensional image feature information. In the information encoding process, the encoder is responsible for converting the input image detection information into a specific encoding format to facilitate the subsequent storage, transmission, or processing of image information. The iteration process requires repeating a certain set of instructions or steps, and each execution is based on the results of the previous iteration to update and gradually approach the final target or result. In the image compression process, the output result of the encoder is the data after compression of the original image, and the above data occupies less space when transmitted or stored, but still obtains key information of the image after decoding.

[0029]

[0030] ​​​In the architecture of the convolutional neural network, 64 convolutional kernels with a size of 3*3 are configured, which means that the network layer has 64 independent convolutional filters, each filter has a spatial size of 3*3 pixels, and each convolutional kernel learns different image information. In this embodiment, different levels of functions correspond to a specific set of convolution functions and corresponding neuron weight coefficients. The above weight coefficients are automatically learned by the network during the training process, which can ensure that each function unit can effectively respond to specific feature information and patterns in the input image information.

[0031] The value of the encoding compression normalization factor is limited between 0 and 1, which is one of the key parameters in the encoding compression process. Through the two-dimensional image compression storage model, the deep extraction and efficient encoding compression of two-dimensional image data features are realized. The above process not only effectively reduces the database size of the device, but also retains the key information in the driving monitoring process, further ensuring the effective operation of the two-dimensional image storage information library.

[0032] Further, the acquisition method of the two-dimensional image storage information library in this embodiment is only an optional condition of the present application. In other one or some embodiments, the acquisition method of the two-dimensional image storage information library can be adjusted according to the collective evaluation conditions of fatigue driving and information storage requirements. By selecting the most suitable image information acquisition method, the computing resources and storage space can be more effectively utilized to reduce the operating cost of the device.

[0033] S2, a two-dimensional image feature recognition model and a two-dimensional image processing model are set in the image information recognition and processing module, and the two-dimensional image storage information library is recognized and processed based on the two-dimensional image feature recognition model and the two-dimensional image processing model to obtain the face monitoring information in the driving process. The specific implementation content is as follows: First, a two-dimensional image feature recognition model is established based on the above binary network.

[0034] The output result of the two-dimensional image compression storage model is identified and analyzed by using the binary network, and the binary network structure is further determined. In the embodiment, a single-layer ConvLSTM recurrent network is used as the binary network, which is internally configured with a single type of convolution kernel with a size of 1*1 and a number of 32. The output result of the information encoder is processed by binary mapping, and the two-dimensional image feature is converted by binary mapping.

[0035] The two-dimensional image feature recognition model satisfies the following relationship: wherein, represents the binary mapping result of the two-dimensional compressed image feature data, an auxiliary constant in a two-dimensional image feature recognition process, a binary mapping function, an output result of an information encoder in an nth iteration process, a neuron weight coefficient in a single-layer ConvLSTM recurrent network structure, a numerical adjustment term for representing a binary mapping.

[0036] In the binary mapping process, the relevant image feature data is converted into a numerical set with a specific range of values, and the above range of values is set to [−1, 1].

[0037] The auxiliary constant refers to the slope, intercept or other properties of the mapping function used to adjust the mapping function in the binary mapping process to control the mapping of data feature values into the target range [−1, 1], and its value range is set to [0, 5] in the embodiment.

[0038] The binary mapping function refers to a function that maps image data from the original space to the target space. In machine learning and neural networks, binary mapping can be used for feature extraction and dimensionality reduction, mapping high-dimensional data to binary space, which can reduce the complexity of data while preserving key information, helping to improve the running efficiency of the device, thereby reducing the computational complexity and storage space requirements of the neural network.

[0039] Through the above binary network, the binary mapping processing of compressed image feature data can be completed, and after the binary mapping processing of the two-dimensional image feature recognition model, the corresponding encoding file of the two-dimensional image data can be obtained, which lays the foundation for the generation of the final encoding compressed file. Therefore, not only the effective extraction and compression of two-dimensional image features are realized, but also the data representation form is further simplified through binary processing, providing a more accurate data basis for subsequent recognition and analysis of two-dimensional images.

[0040] Then, a two-dimensional image processing model is established based on the above decoder network.

[0041] In this embodiment, the running mechanism of the decoder network is opposite to that of the encoder network, and the original three-channel (RGB) two-dimensional image data is restored according to the adjusted encoding compressed file. The two-dimensional image processing model is composed of multiple layers, mainly including two layers of convolutional neural network, four layers of recurrent network and deep compression layer.

[0042] In the decoding process, each layer of the recurrent network processes the image feature information it receives and outputs an updated feature map. Then, the feature map is input into a deep compression layer for further processing. The deep compression layer can reorganize the compressed encoding information of the two-dimensional image, aggregate and expand the two-dimensional image information from different depth levels to a unified depth level. This step not only helps to effectively integrate the two-dimensional image, but also reduces the number of encoding information channels, thereby reducing the computational complexity of the two-dimensional image decoding process.

[0043] Finally, through the layer-by-layer processing and information integration of the two-dimensional image processing model, the restored two-dimensional image can be obtained. The encoding compressed file is input into the processing model, and after multiple convolution, recurrent network and deep compression processing, the restored two-dimensional image data is finally output.

[0044] The two-dimensional image processing model satisfies the following relationship: wherein, represents the restored two-dimensional image data, represents the corresponding weight coefficient, represents the first neuron function of the convolutional neural network, represents the information set after image feature binary mapping, represents the auxiliary parameter of the deep compression layer, represents the first neuron function of the recurrent network in the processing model, represents the second neuron function of the recurrent network in the processing model, represents the third neuron function of the recurrent network in the processing model, represents the fourth neuron function of the recurrent network in the processing model, represents the corresponding weight coefficient, represents the corresponding weight coefficient, represents the corresponding weight coefficient, represents the corresponding weight coefficient, represents the corresponding weight coefficient, represents the second neuron function of the convolutional neural network.

[0045] In an optional embodiment, the two-dimensional image feature recognition model is used to recognize the two-dimensional image storage information library to obtain a feature mapping result of the two-dimensional compressed image; the feature mapping result is processed by a two-dimensional image processing model to obtain restoration information of a two-dimensional feature image, and based on the feature mapping result and the two-dimensional feature image restoration information, face monitoring information in the driving process can be accurately obtained.

[0046] In the embodiment, the two-dimensional image feature recognition model recognizes the images stored in the two-dimensional image information library, and then extracts the feature mapping result of the two-dimensional compressed image, thereby providing key information for subsequent processing.

[0047] Then, the feature mapping result is deeply processed by the two-dimensional image processing model, and key information in the two-dimensional feature image is restored based on this, thereby laying a foundation for the actual application of the face positioning and fatigue driving detection device.

[0048] Finally, based on the feature mapping result and the two-dimensional feature image restoration information, face monitoring information in the driving process can be obtained, and through the encoder network, the binarization network and the decoder network, lossless compression and decompression of two-dimensional image data in the driving process are realized, which not only significantly improves the storage efficiency of the two-dimensional image data, but also provides strong support for the rapid transmission of monitoring data.

[0049] Further, the way of obtaining the face monitoring information in the driving process in the embodiment is only an optional condition of the present application, and in other one or some embodiments, the specific structure of the image information recognition and processing module and the monitoring information target can be adjusted according to the specific structure of the image information recognition and processing module and the monitoring information target. In actual application, the specific structure of the image information recognition and processing module and the monitoring target will be different under different detection scenes and requirements, and adaptive adjustment of the process of obtaining the face monitoring information can ensure the accuracy and real-time performance of the monitoring information, and improve the operation efficiency and stability of the entire device.

[0050] S3, a face analysis model and a fatigue driving evaluation system are established in the face analysis and fatigue driving evaluation module, the face analysis and fatigue driving evaluation module receives and processes the face monitoring information, and outputs a fatigue driving evaluation result in the driving process, and the specific implementation contents are as follows: In an optional embodiment, a face analysis model is established based on the fatigue driving detection requirement, and the face analysis model mainly includes a face recognition algorithm, an eye recognition algorithm, a two-dimensional pan-tilt control algorithm, a PERCLOS algorithm and a blink frequency algorithm.

[0051] 1. Face recognition algorithm In this embodiment, the Haar feature classifier on OpenMV is selected. The above classifier has real-time analysis performance and low algorithm requirement. Considering the challenges such as limited algorithm power faced by the vehicle-mounted electronic system, the Haar feature classifier can quickly and accurately identify the two-dimensional monitoring image.

[0052] Using the Haar feature classifier on the OpenMV platform, the rapid recognition of the face in the two-dimensional monitoring image range in the face monitoring information is realized. This algorithm can efficiently process image data, accurately locate the position of the face, and output the coordinate information of the face , which provides technical support for subsequent two-dimensional image processing, face tracking, and face recognition applications.

[0053] 2. Eye recognition algorithm During the driving process of the vehicle, the output result based on the face recognition algorithm can further determine whether the face position of the driver presents a relatively fixed state. In the embodiment, the face recognition technology is used to recognize the eyes, and the Haar operator is used to efficiently and accurately extract the face features, thereby quickly recognizing the face region of the driver.

[0054] On the basis of successfully recognizing the face region of the driver, the Cascade classifier of the Haar feature is further used. After training, the above classifier can accurately detect the eye position in the face, based on which the eye region of the driver can be effectively marked, and the pupil position and coordinate information of the driver can be further obtained.

[0055] In this embodiment, the Haar operator and the Cascade classifier are combined to recognize and analyze the face monitoring information during the driving process, that is, to effectively recognize the face region and eye position of the driver, which provides strong support for subsequent fatigue driving detection and other applications.

[0056] 3. Two-dimensional pan-tilt control algorithm The above two-dimensional pan-tilt is a precise mechanical device composed of precise servos, which can carry the OpenMV camera to realize 360° omnidirectional rotation in the horizontal direction. The physical assembly diagram is shown in Figure 2 .

[0057] In this embodiment, the face monitoring information during the driving process is recognized and analyzed by using OpenMV, and the face coordinate information of the driver is accurately obtained Subsequently, by calculating the center point coordinates of the detection frame, the position of the driver in the two-dimensional image can be accurately located, and when the position of the center point deviates beyond a pre-set threshold, the two-dimensional gimbal control algorithm immediately calculates the size of the deviation and controls the direction adjustment of the gimbal accordingly. Based on this, the camera always maintains high consistency with the geometric center of the driver's face during driving, regardless of how the driver moves.

[0058] In this key reference system of pixel coordinates, the complex relationship between the driver center of the gimbal and the rotation of the gimbal can be analyzed in depth, and the turning direction of the gimbal can be further accurately divided into eight different cases. For details, please refer to Figure 3 wherein numbers 1 to 8 represent the eight cases of the turning direction of the gimbal, and xoy represents the rotation coordinate axis of the two-dimensional gimbal.

[0059] In the selection of the two-dimensional gimbal control algorithm, the positional PID algorithm can integrate the deviation of the entire control algorithm, but each output value is closely related to the motion state of the previous frame object, which may lead to an increase in error accumulation. In this embodiment, the incremental PID algorithm is used, which compares the error values of three consecutive frames to obtain the final output value, thereby avoiding error accumulation and ensuring the control accuracy of the two-dimensional gimbal.

[0060] In summary, the two-dimensional gimbal control algorithm in the embodiment ingeniously combines the OpenMV camera, the control gimbal, and the incremental PID control algorithm, realizes real-time tracking and accurate analysis of the driver's face, and provides technical support for the practical application of the face positioning and fatigue driving detection device.

[0061] 4. PERCLOS algorithm The eye changes of the driver are an important indicator for evaluating the fatigue state. The PERCLOS algorithm can quantify the proportion of time that the driver's eyes are closed to a certain degree within a certain time period, and then judge whether the driver is in a fatigue state during driving.

[0062] The PERCLOS algorithm in the embodiment defines various general standards, including but not limited to P70, P80, and EM, to more comprehensively and accurately evaluate the degree of fatigue. The core of the above standards is to calculate the proportion of time that the eye closure exceeds the pupil by 70%, 80%, and 50% of the total observation time within a given time interval. In an alternative embodiment, the P70 standard means that the proportion of time that the eye closure exceeds the pupil by 70%; the P80 and EM standards correspond to the closure degree thresholds of 80% and 50%, respectively.

[0063] By combining the PERCLOS algorithm and facial monitoring information to quantify the degree of eye closure, an objective and accurate assessment of driver fatigue can be achieved, providing a strong guarantee for safe driving.

[0064] 5. Blink frequency algorithm Blink frequency algorithms can compare the changes in pupil area between normal driving and fatigued driving states to detect the number and frequency of blinks. These algorithms can accurately identify and accumulate each blink. Taking the P80 standard as an example, the algorithm only records a blink when the eye closure reaches or exceeds a threshold of 80%. This simplifies the blink recognition process and reduces the hardware requirements, making it more suitable for in-vehicle electronic systems.

[0065] Blink frequency algorithms can effectively monitor a driver's blink frequency, thereby quickly assessing the driver's fatigue state. In one optional embodiment, the blink frequency algorithm is used to analyze the blink frequency over a driving period. During the operation of the blink frequency algorithm, the P80 standard is followed, meaning that only when the degree of eye closure reaches or exceeds the threshold of 80% is the action considered a valid blink and the data is recorded. In this way, the total number of blinks can be counted within a specific time period, and the ratio of the total number of blinks to the time is the blink frequency.

[0066] Furthermore, a schematic diagram of the blink frequency detection algorithm within a specific time period is provided; please refer to [link / reference]. Figure 4 .based on Figure 4 It can be seen that the above blink frequency algorithm satisfies the following relationship: in, Indicates blink frequency, express Total number of blinks during the monitoring period Indicates the end time of the monitoring period. Indicates the start time of the monitoring period.

[0067] The design of the above-mentioned blink frequency algorithm not only improves the accuracy of blink recognition, but also ensures the reliability of the blink frequency calculation results. It can effectively monitor the driver's blinking behavior and provide strong support for face positioning and fatigue driving detection devices.

[0068] In another optional embodiment, a fatigue driving evaluation system is established based on the analysis results of the face analysis model and the two-dimensional image storage information library. The fatigue driving evaluation system mainly includes the face geometric center condition, the eye closure state, the blink frequency, the head posture stability, the facial expression recognition, and the driving duration and the rest time.

[0069] In the embodiment, a fatigue driving evaluation system based on the face analysis model and the two-dimensional image storage information library is constructed. The system covers multiple key dimensions, which is helpful for more comprehensive and accurate evaluation of the fatigue state of the driver. The specific content is as follows: The geometric center coordinates of the driver's face are obtained through the face recognition algorithm which is helpful for real-time monitoring and analysis of the overall dynamics of the driver's face, thereby indirectly reflecting the driving state in the driving process.

[0070] Combined with the eye recognition algorithm and the PERCLOS algorithm, the closure degree and change state of the driver's eyes can be quickly obtained and analyzed. The evaluation index is helpful for accurately judging whether the driver is in a fatigue state.

[0071] The blink frequency is also one of the important indicators for evaluating the fatigue degree of the driver. In the embodiment, the blink frequency algorithm is used to quickly calculate the number of blinks of the driver in the driving process, thereby further analyzing the fatigue state. On the other hand, in addition to the blink frequency, the speed and force of the blink are also important indicators reflecting the fatigue degree of the driver. For example, when tired, the blink may become slow or weak.

[0072] At the same time, the head posture stability evaluation index is also considered. By analyzing the tilt angle and rotation angle of the driver's head, the stability of the head posture in the driving process can be evaluated, thereby indirectly reflecting the attention concentration degree and the fatigue state.

[0073] In terms of facial expression recognition, advanced facial expression recognition technology is used in the embodiment to capture the subtle expression changes of the driver in the driving process, such as frowning, yawning, and blinking. The driver's expression is often closely related to the state of fatigue, drowsiness, and other states, and is also an important basis for evaluating the fatigue degree of the driver.

[0074] Finally, the time stamp information in the two-dimensional image storage information library is combined to calculate the continuous driving duration of the driver and the corresponding rest time. Through the above information, it can be evaluated whether the driver meets the reasonable driving and rest cycle, which can effectively prevent fatigue driving.

[0075] The fatigue driving evaluation system of the embodiment comprehensively considers information in multiple dimensions, can more comprehensively and accurately evaluate the fatigue state of the driver, and establishment of the system not only provides a powerful evaluation index for driving safety, but also provides a reference basis for practical application of the two-dimensional pan-tilt face positioning and fatigue driving detection device.

[0076] The face analysis and fatigue driving evaluation module outputs a fatigue driving evaluation result in the driving process in combination with the face analysis result and the fatigue driving evaluation system.

[0077] The face analysis and fatigue driving evaluation module integrates multiple algorithms, including but not limited to a face recognition algorithm, an eye recognition algorithm, a two-dimensional pan-tilt control algorithm, a PERCLOS algorithm, and a blink frequency algorithm, which is helpful for comprehensive analysis of face features in the driving process and output of a fatigue driving evaluation result.

[0078] The module first accurately recognizes facial features of the driver by using the face recognition algorithm, and then further positions and analyzes the eye state of the driver by using the eye recognition algorithm and the two-dimensional pan-tilt control algorithm. On this basis, the PERCLOS algorithm is used to accurately measure the eyelid closure time ratio, and the blink frequency algorithm is used to count the number of blinks and characteristics of the driver in the driving process.

[0079] In combination with the output results of the related algorithms, the fatigue driving evaluation system can further evaluate the fatigue degree of the driver. This includes but is not limited to: the eye closure degree and the blink frequency to judge the alertness of the driver; the head posture and facial expression analysis to indirectly reflect the attention concentration degree and fatigue state of the driver; and analysis of the eye movement trajectory to understand the line of sight direction and attention allocation of the driver.

[0080] Finally, the face analysis and fatigue driving evaluation module comprehensively sorts and summarizes the above analysis results, uses preset evaluation standards and logic to output a comprehensive and accurate fatigue driving evaluation result, and provides technical support for actual operation and wide application of the two-dimensional pan-tilt face positioning and fatigue driving detection device.

[0081] Further, the specific analysis steps of the fatigue driving evaluation result in the embodiment are only an optional condition of the present application, and in other one or some embodiments, the specific analysis steps of the fatigue driving evaluation result can be changed and optimized according to the safety driving recognition requirements and platform monitoring conditions, so that the device of the present application can better adapt to the driving safety requirements in different scenarios, improve the practicality and flexibility of the device, and promote the continuous development and progress of the driving detection technology.

[0082] S4, the intelligent alarm module sends a fatigue warning signal in real time during driving according to the fatigue driving evaluation result, and realizes real-time monitoring and early warning of the fatigue driving state based on the fatigue warning signal, and the specific implementation content is as follows: A two-dimensional holder face positioning and fatigue driving detection device further comprises a safety warning device and a remote communication device configured in the alarm module; the fatigue driving state during driving is detected and intelligently warned in real time by combining the safety warning device, the remote communication device and the fatigue driving evaluation result, so as to realize real-time monitoring and early warning of the fatigue driving state.

[0083] During driving, the intelligent alarm module sends a fatigue warning signal to the driver in a timely and effective manner according to the fatigue driving evaluation result output by the image information recognition and processing module in step S3, and the above function is mainly realized by comprehensively analyzing the key information such as the facial features, eye movements and head posture of the driver, and further constructing a real-time monitoring and early warning system for the fatigue driving state.

[0084] A two-dimensional holder face positioning and fatigue driving detection device not only has face positioning and analysis capabilities, but also integrates an intelligent alarm module. The safety warning device and the remote communication device are configured in the module. The above devices work cooperatively, and in combination with real-time data of fatigue driving evaluation, the fatigue driving state during driving is monitored and intelligently warned in real time.

[0085] The safety warning device can receive the fatigue driving evaluation result and the evaluation system analysis result of the driver in real time, including but not limited to frequent blinking, wandering eyes, frequent head shaking or long-time fixed posture, etc. Once the relevant signs of fatigue are identified, the remote communication device will quickly start and send the warning information to the designated receiving end, including but not limited to the driver's mobile phone, the vehicle-mounted display screen or the monitoring center. Whether the driver himself or the relevant management personnel can receive the fatigue driving warning signal in the first time, so as to take corresponding measures to avoid potential safety hazards.

[0086] In addition, the accuracy and real-time performance of the fatigue driving detection result can be further improved by continuously optimizing the algorithm and improving the performance of the related devices in the device. At the same time, the driving habits of the driver are further mined and analyzed by combining big data analysis technology, which provides strong support for formulating more personalized fatigue driving prevention strategies. Through relevant measures, the fatigue driving state of the driver is comprehensively, real-time and intelligently monitored, and the road traffic safety is protected.

[0087] Please refer to Figure 5In an optional embodiment, the application further provides a two-dimensional holder face positioning and fatigue driving detection device, which comprises a data collection and storage module, an image information recognition and processing module, a face analysis and fatigue driving evaluation module and an intelligent alarm module, the data collection and storage module, the image information recognition and processing module, the face analysis and fatigue driving evaluation module and the intelligent alarm module are connected with each other, and the specific steps of the two-dimensional holder face positioning and fatigue driving detection device are implemented according to the embodiments of the application. The two-dimensional holder face positioning and fatigue driving detection device has a complete structure, is objective and stable, and improves the overall applicability and practical application ability of the application.

[0088] Finally, it should be noted that: the above embodiments are only used to illustrate the technical solutions of the application, but not to limit them; although the application has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that: it can still modify the technical solutions recorded in the foregoing embodiments, or make equivalent replacement for part or all of the technical features; and these modifications or replacements do not make the essence of the corresponding technical solutions deviate from the scope of the technical solutions of the embodiments of the application, and they should be covered in the scope of the claims and the description of the application.

Claims

1. A two-dimensional holder face positioning and fatigue driving detection device, characterized in that, The two-dimensional gimbal face positioning and fatigue driving detection device includes: a data collection and storage module, an image information recognition and processing module, a face analysis and fatigue driving assessment module, and an intelligent alarm module; A two-dimensional image lossless storage mechanism is established in the data collection and storage module. Based on the two-dimensional image lossless storage mechanism, facial image information during driving is compressed and stored, and a two-dimensional image storage information database during driving is obtained. A two-dimensional image feature recognition model and a two-dimensional image processing model are set in the image information recognition and processing module. Based on the two-dimensional image feature recognition model and the two-dimensional image processing model, the two-dimensional image storage information database is recognized and processed to obtain face monitoring information during driving. A face analysis model and a fatigue driving assessment system are established in the face analysis and fatigue driving assessment module. The face analysis and fatigue driving assessment module receives and processes the face monitoring information and outputs the fatigue driving assessment results during the driving process. The intelligent alarm module sends a fatigue warning signal in real time during driving based on the fatigue driving assessment results, and realizes real-time monitoring and early warning of fatigue driving status based on the fatigue warning signal.

2. The two-dimensional holder face positioning and fatigue driving detection device according to claim 1, characterized in that, The establishment of a two-dimensional image lossless storage mechanism in the data collection and storage module includes: Facial image information during driving is obtained through the data collection and storage module; Based on the information requirements of the fatigue driving detection device and the facial image information, a two-dimensional image compression and storage model is established in the two-dimensional image lossless storage mechanism. 3.The two-dimensional holder face positioning and fatigue driving detection device according to claim 2, characterized in that, The step of compressing and storing facial image information during driving based on the lossless storage mechanism for two-dimensional images, and obtaining a two-dimensional image storage information database during driving, includes: The two-dimensional image lossless storage mechanism compresses and stores the face image information through the two-dimensional image compression storage model to obtain a two-dimensional image storage information library during the driving process; The two-dimensional image compression and storage model satisfies the following relationship: wherein, represents the output result of the information encoder in the nth iteration process, represents a first function corresponding to the ConvLSTM recurrent network, represents a second function corresponding to the ConvLSTM recurrent network, represents a third function corresponding to the ConvLSTM recurrent network, represents a corresponding neuron weight coefficient, represents a corresponding neuron weight coefficient, represents a corresponding neuron weight coefficient, represents a normalization factor of the encoding compression, represents a convolutional neural network function, represents an initial two-dimensional image of the driver in the driving process, represents an influence weight of the accumulation neural network function.

4. The two-dimensional holder face positioning and fatigue driving detection device according to claim 1, characterized in that, The step of setting a two-dimensional image feature recognition model and a two-dimensional image processing model in the image information recognition and processing module includes: A two-dimensional image feature recognition model is established based on a binary recurrent network structure; The two-dimensional image feature recognition model satisfies the following relationship: wherein, represents the binarization mapping result of the two-dimensional compressed image feature data, represents an auxiliary constant in the two-dimensional image feature recognition process, represents a binarization mapping function, represents the output result of the information encoder in the nth iteration process, represents the neuron weight coefficient in the single-layer ConvLSTM recurrent network structure, represents a numerical adjustment term representing binarization mapping.

5. The two-dimensional holder face positioning and fatigue driving detection device according to claim 4, characterized in that, The step of setting a two-dimensional image feature recognition model and a two-dimensional image processing model in the image information recognition and processing module includes: A two-dimensional image processing model is established based on a decoder neural network structure; The two-dimensional image processing model satisfies the following relationship: wherein, represents the restored two-dimensional image data, represents corresponding weight coefficients, represents a first neuron function of the convolutional neural network, represents an information set after image feature binarization mapping, represents an auxiliary parameter of the deep compression layer, represents a first neuron function of the recurrent network in the processing model, represents a second neuron function of the recurrent network in the processing model, represents a third neuron function of the recurrent network in the processing model, represents a fourth neuron function of the recurrent network in the processing model, represents corresponding weight coefficients, represents corresponding weight coefficients, represents corresponding weight coefficients, represents corresponding weight coefficients, represents corresponding weight coefficients, represents a second neuron function of the convolutional neural network.

6. The two-dimensional holder face positioning and fatigue driving detection device according to claim 4, characterized in that, The step of recognizing and processing the two-dimensional image storage information database based on the two-dimensional image feature recognition model and the two-dimensional image processing model to obtain face monitoring information during driving includes: The two-dimensional image feature recognition model is used to identify the two-dimensional image storage information database to obtain the feature mapping result of the two-dimensional compressed image; The feature mapping result is processed by the two-dimensional image processing model to obtain two-dimensional feature image restoration information; Based on the feature mapping results and the two-dimensional feature image restoration information, face monitoring information during driving is obtained.

7. The two-dimensional holder face positioning and fatigue driving detection device according to claim 1, characterized in that, The face analysis model and the fatigue driving evaluation system are established in the face analysis and fatigue driving evaluation module, and the face analysis model and the fatigue driving evaluation system include: The face analysis model is established based on the fatigue driving detection requirement, and the face analysis model includes a face recognition algorithm, an eye recognition algorithm, a two-dimensional pan-tilt control algorithm, a PERCLOS algorithm, and a blink frequency algorithm. 8.The two-dimensional holder face positioning and drowsy driving detection device according to claim 7, characterized in that, The face analysis model and the fatigue driving evaluation system are established in the face analysis and fatigue driving evaluation module, and the face analysis model and the fatigue driving evaluation system include: The fatigue driving evaluation system is established based on the analysis result of the face analysis model and a two-dimensional image storage information base, and the fatigue driving evaluation system includes a face geometric center condition, an eye closing state, a blink frequency, a head posture stability, a facial expression recognition, and a driving time and a rest time. 9.The two-dimensional holder face positioning and drowsy driving detection device according to claim 8, characterized in that, The face analysis and fatigue driving evaluation module receives and processes the face monitoring information and outputs a fatigue driving evaluation result in a driving process, and the face analysis and fatigue driving evaluation module includes: The face analysis and fatigue driving evaluation module obtains a face analysis result in a driving process through the face recognition algorithm, the eye recognition algorithm, the two-dimensional pan-tilt control algorithm, the PERCLOS algorithm, and the blink frequency algorithm; The face analysis and fatigue driving evaluation module outputs a fatigue driving evaluation result in a driving process in combination with the face analysis result and the fatigue driving evaluation system. 10.The two-dimensional holder face positioning and drowsy driving detection device according to claim 9, characterized in that, The intelligent alarm module sends a fatigue warning signal in a driving process in real time according to the fatigue driving evaluation result, and real-time monitoring and early warning of a fatigue driving state are realized based on the fatigue warning signal, and the intelligent alarm module includes: A safety warning device and a remote communication device are configured in the alarm module; Real-time detection and intelligent early warning of a fatigue driving state in a driving process are realized in combination with the safety warning device, the remote communication device, and the fatigue driving evaluation result, so as to realize real-time monitoring and early warning of a fatigue driving state.

Images