A method and system for determining whether a driver is sneezing

Abstract

The application discloses a method and system for judging whether a driver sneezes or not, and the method comprises the following steps: training a neural network model by using a database to obtain a trained neural network model, wherein the database comprises sound feature data, eye feature data and mouth feature data of the driver when sneezing; processing the real-time collected eye feature data, mouth feature data and sound feature data of the driver by using the trained neural network model to obtain feature data corresponding to the eye feature, mouth feature and sound feature of the driver respectively; and judging whether the driver sneezes or not according to the feature data corresponding to the eye feature, mouth feature and sound feature of the driver. The application can detect and identify the sneezing behavior of the driver during driving, and the detection and identification results of the application can provide data support for the auxiliary driving system of the automobile, and are helpful for further improvement of the auxiliary driving system and safety guarantee during driving.

Description

Technical Field

[0001] This invention belongs to the field of automotive safe driving technology, specifically relating to a method and system for determining whether a driver has sneezed. Background Technology

[0002] With the development of technology, driving has become an increasingly common mode of transportation. However, this has also led to a rise in traffic hazards, making traffic safety a pressing issue that urgently needs to be addressed. Road traffic accidents are often caused by a combination of factors, including people, vehicles, and roads. Studies have shown that human factors account for up to 90% of driving accidents, with improper driving behavior being a major contributing factor. Drivers, as key participants in road traffic, act as decision-makers, traffic controllers, and information processors. Their behavior has a significant impact on the entire traffic system. Therefore, safe driving behavior is a crucial component of modern traffic science research.

[0003] Numerous reports, both domestic and international, have documented instances of drivers involuntarily sneezing while driving due to an itchy nose or pre-existing rhinitis, leading to a series of traffic accidents. Because sneezing causes a brief loss of control, if the vehicle's speed is not promptly controlled or further accelerated, an accident is highly likely. Therefore, sneezing while driving poses a significant traffic safety hazard. Sneezing is a reflexive physiological phenomenon that is difficult for humans to suppress. For most people, sneezing is insignificant regardless of time, place, or occasion, but for motor vehicle drivers, it hides a very dangerous accident-causing factor. Medical tests show that the entire process of a normal person's sneeze takes approximately 10-15 seconds, and may take longer in some cases. If a driver is traveling at 50 kilometers per hour, a sneeze can cover at least 150 meters by the time the vehicle has traveled, while the normal braking distance at 100 kilometers per hour is only about 40 meters. In a sudden situation, there is simply not enough time to avoid a collision within 150 meters. When a person sneezes, they unconsciously close their eyes and open their mouth wide. At this time, the driver cannot see the road clearly, hear external sounds, and their brain and limbs become unresponsive, easily leading to a traffic accident. This is even more dangerous for people with colds or allergic rhinitis. Therefore, if a system could provide early warnings of impending sneezing while driving, accurately identify the driver's sneezing behavior, and transmit this signal to the car's infotainment system to promptly implement safe driving control strategies, the occurrence of accidents caused by this reason could be reduced.

[0004] In existing technologies, researchers both domestically and internationally have made considerable efforts to study driver behavior and states. International researchers have established relatively complete driving behavior models based on driving experiments, achieving certain results and applying their findings to the intelligent design of driver assistance systems and human-vehicle coupling technologies. Domestic researchers, targeting different driving behaviors, have used driving simulators as experimental foundations for qualitative and quantitative analysis, studying monitoring indicators and methods for safe driving behavior based on vehicle operation, road construction, and driver state. However, there is currently no technology for monitoring unsafe behaviors such as sneezing during driving. Research on monitoring indicators and methods for safe driving behavior only defines the research scope and monitoring space for safe driving behaviors during driving, without actually solving the problem of monitoring specific unsafe driving behaviors. Human-vehicle coupling-based safe driving behavior monitoring technologies also lack detection and warning for sneezing, and these technologies require drivers to wear detection devices, leading to significant uncertainty in driver acceptance. Therefore, detecting and identifying sneezing behavior during driving is essential. Summary of the Invention

[0005] To address the problems existing in the prior art, the present invention aims to provide a method and system for determining whether a driver is sneezing. The present invention can detect and identify sneezing behavior of a driver while driving. The detection and identification results of the present invention can provide data support for the vehicle's driver assistance system, which helps to further improve the driver assistance system and ensure driving safety.

[0006] To achieve the above objectives, the present invention adopts the following technical solution:

[0007] One method for determining whether a driver has sneezed includes the following steps:

[0008] A neural network model is trained using a database to obtain a trained neural network model. The database includes sound feature data, eye feature data, and mouth feature data of a driver sneezing.

[0009] The trained neural network model is used to process the real-time collected driver's eye feature data, mouth feature data, and voice feature data to obtain the feature data corresponding to the driver's eye feature, mouth feature, and voice feature respectively.

[0010] Determine whether the driver sneezed based on the characteristic data corresponding to the driver's eye features, mouth features, and voice features.

[0011] Preferably, the feature data corresponding to the eye features includes data containing eye image features and eye closure duration data, the feature data corresponding to the mouth features includes data containing mouth opening degree data, and the feature data corresponding to the voice features includes data containing volume, frequency, and duration data.

[0012] Preferably, the process of extracting the eye image features includes:

[0013] The original images of the eyes are subjected to image transformation, image encoding compression, image enhancement and restoration, and image segmentation to obtain two-dimensional image data of the eyes. The two-dimensional image data of the eyes is then used as the features of the eye images.

[0014] Preferably, the data extraction process for the duration of eye closure includes:

[0015] The collected data on the duration of eye closure is integrated and converted into a one-dimensional vector of time data, which is then used as the duration of eye closure data.

[0016] Preferably, the process of extracting the degree of mouth opening includes:

[0017] The original images of the mouth are subjected to image transformation, image encoding compression, image enhancement and restoration, and image segmentation to obtain two-dimensional image data of the mouth. The two-dimensional image data of the mouth is used as the mouth image feature, which includes the feature of the degree of mouth opening.

[0018] Preferably, the process of extracting the sound feature data includes:

[0019] The collected audio is noise-reduced and compressed, and then the volume, frequency and duration of the audio are extracted. Finally, the extracted volume, frequency and duration are graded and quantized to obtain a one-dimensional vector, which is used as the sound feature data.

[0020] Preferably, the process of using a trained neural network model to process the real-time collected driver's eye feature data, mouth feature data, and voice feature data to obtain the corresponding feature data for the driver's eye features, mouth features, and voice features includes:

[0021] The trained neural network model is used to process eye image features to determine whether the driver's eyes are closed.

[0022] The trained neural network model is used to process the data on the duration of eye closure to determine whether the duration of eye closure reaches the preset duration of eye closure when sneezing.

[0023] The trained neural network model is used to process data including the degree of mouth opening to determine whether the degree of mouth opening is equal to the degree of mouth opening when the driver sneezes.

[0024] The trained neural network model is used to process the sound feature data to determine whether the sound emitted by the driver is a sneezing sound.

[0025] Preferably, the process of determining whether a driver has sneezed based on the characteristic data corresponding to the driver's eye features, mouth features, and voice features includes:

[0026] If the driver closes their eyes, the duration of eye closure reaches the preset duration of eye closure for sneezing, the degree of mouth opening reaches the degree of mouth opening for sneezing, and the sound emitted by the driver is the "ah-choo" sound of sneezing, then it is determined that the driver sneezed at this time; otherwise, it is determined that the driver did not sneeze.

[0027] The present invention also provides a system for determining whether a driver has sneezed, comprising:

[0028] Eye feature data acquisition unit: used to record driver's eye movement information in real time and extract driver's eye feature data from the driver's eye movement information;

[0029] Mouth feature data acquisition unit: used to record the driver's mouth movement information in real time and extract the driver's mouth feature data from the driver's mouth movement information;

[0030] The sound feature data acquisition unit is used to collect the sound emitted by the driver in real time and extract sound feature data from the sound emitted by the driver.

[0031] Data processing unit: Used to process real-time collected driver eye feature data, mouth feature data and voice feature data using a neural network model, to obtain feature data corresponding to the driver's eye features, mouth features and voice features respectively.

[0032] Judgment module: Used to determine whether the driver has sneezed based on the feature data corresponding to the driver's eye features, mouth features, and voice features.

[0033] Preferably, the neural network model includes a two-dimensional convolutional neural network model and a one-dimensional convolutional neural network model;

[0034] When determining whether the driver's eyes are closed, the two-dimensional convolutional neural network model is used to process the eye image features;

[0035] When determining whether the duration of eye closure reaches the preset duration of eye closure when sneezing, the one-dimensional convolutional neural network model is used to process the eye closure duration data.

[0036] When determining whether the degree of mouth opening is equal to the degree of mouth opening when a driver sneezes, the two-dimensional convolutional neural network model is used to process the data containing the degree of mouth opening.

[0037] The trained neural network model is used to process the sound feature data to determine whether the sound emitted by the driver is the sneezing sound.

[0038] The basic structure of the two-dimensional convolutional neural network includes an input layer, six convolutional blocks, three fully connected layers, and an output layer. The input layer has a data size of 224*224, 3 channels, and each element represents a grayscale value. Each convolutional block contains one convolutional layer and one pooling layer. Each convolutional layer contains 100 convolutional kernels of size 3*3 with a stride of 2. The pooling layer uses max pooling, with a pooling region of size 3*3. After the hidden features are extracted by the convolutional blocks, the element matrix is ​​flattened and fed into the fully connected layer. A Dropout layer is added to the fully connected layer, using ReLU as the activation function and MSE as the objective function. The output layer consists of two neurons, with 0 indicating no and 1 indicating yes.

[0039] The basic structure of the one-dimensional convolutional neural network includes an input layer, six convolutional blocks, three fully connected layers, and an output layer. The input layer receives 128*1 data points, which are one-dimensional vectors. Each convolutional block contains a convolutional layer and a pooling layer. The convolutional layer has 80 kernels of size 3*1 with a stride of 3. The pooling layer uses max pooling with a pooling region of size 2*1. After the hidden features are extracted by the convolutional blocks, the element matrix is ​​flattened and fed into the fully connected layer. A Dropout layer is added to the fully connected layer, using ReLU as the activation function and MSE as the objective function. The output layer consists of two neurons, with 0 indicating no and 1 indicating yes.

[0040] Compared with the prior art, the present invention has the following beneficial effects:

[0041] This invention addresses the impact of uncontrollable sneezing on driving safety. It processes driver eye, mouth, and voice feature data collected during driving using a neural network model. The processing results determine whether the driver has sneezed. Eye feature data can be collected using existing infrared sensors, mouth feature data using infrared cameras, and voice feature data using conventional sound monitoring equipment. Therefore, this method eliminates the need for the driver to wear monitoring devices or sensors, allowing for sneezing monitoring during driving without affecting visibility or comfort. It is easily accepted by drivers, provides data support for automotive driver assistance systems, contributes to the further improvement of these systems, and ensures safer driving. Attached Figure Description

[0042] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are merely embodiments of the present invention. For those skilled in the art, the schematic diagram of this patent cannot be obtained without creative effort.

[0043] Figure 1 This is a flowchart of the convolutional neural network training process of this invention;

[0044] Figure 2 This is a flowchart for determining whether a driver has sneezed in this invention. Detailed Implementation

[0045] The following are specific embodiments of the present invention. It should be noted that the present invention is not limited to the following specific embodiments. All equivalent modifications made based on the technical solutions of this application fall within the protection scope of the present invention.

[0046] Specifically, when sneezing, facial expressions include opening the mouth and closing the eyes, while the sound is "achoo." Corresponding feature data is collected from sneezing samples to establish a database. The established neural network model is then repeatedly trained. A two-dimensional convolutional neural network is used to recognize facial expressions, while a one-dimensional convolutional neural network is used to process time-series sound waves. The improved model is then used to ultimately determine whether the driver has sneezed.

[0047] Specifically, this invention requires the collection of facial and audio data from the driver during implementation. The facial data includes the driver's eye and mouth data.

[0048] Eye Data Acquisition: Infrared eye sensors are used to record blinking feature data. The output includes eye image features and the duration of eye closure. The output eye image features and eye closure duration are processed by the central controller's image processing unit and digital processing unit respectively. The specific image processing unit steps include receiving the raw image, performing image transformation and image encoding compression to save image transmission and processing time and reduce memory usage, then performing image enhancement and restoration to improve image quality and clarity, and finally performing image segmentation to accurately extract the required eye features, thus outputting a 224*224 two-dimensional image with 3 channels. The digital processing unit integrates and converts the input time data, ultimately outputting a 128*1 one-dimensional time vector. The obtained data is then input into a pre-built 2D convolutional neural network and a 1D convolutional neural network to determine whether the driver's eyes are closed and whether the eye closure duration meets the preset eye closure duration for sneezing.

[0049] Mouth data acquisition: An infrared camera is used to record and acquire mouth feature data. The output includes mouth image features. After the output mouth image features are processed by the central controller image processing unit, a two-dimensional image data with a size of 224*224 and 3 channels is obtained. The specific image processing steps are the same as those for eye image data processing. The obtained data is then input into a pre-built two-dimensional convolutional neural network to determine whether the mouth opening degree is equal to that of a sneeze.

[0050] Sound data acquisition: Sound monitoring equipment is used to collect and record the sound characteristics of a sneeze. The output sound characteristic data is processed by the central controller audio processing unit. The specific audio processing steps include first performing noise reduction on the audio to improve recognition accuracy, then compressing the audio to reduce computation and processing time, then extracting the volume, frequency, and duration of the obtained audio, and finally performing hierarchical quantization on the extracted parts to obtain a one-dimensional vector of size 128*1, representing the time series audio data. All the obtained audio data is then input into a pre-built one-dimensional convolutional neural network to determine whether the sound feature is the 'achoo' sound of a sneeze.

[0051] The details of the neural network model used in this invention are as follows:

[0052] The basic structure of a two-dimensional convolutional neural network includes an input layer, a convolutional layer, a pooling layer, a fully connected layer, and an output layer. The input layer contains processed image data with a size of 224*224 and 3 channels, where each element represents a grayscale value. The network structure designed in this invention includes 6 convolutional blocks (each containing one convolutional layer and one pooling layer), three fully connected layers, and one output layer. The convolutional layers contain 100 3*3 convolutional kernels with a stride of 2. The pooling layers use max pooling with a pooling region of 3*3. After the hidden features are extracted by the convolutional blocks, the element matrix is ​​flattened and fed into the fully connected layers. A Dropout layer is added to the fully connected layers to prevent overfitting during network training. The activation function is ReLU, the objective function is MSE, and the output layer consists of two neurons, where 0 indicates no and 1 indicates yes.

[0053] The basic structure of a one-dimensional convolutional neural network includes an input layer, a convolutional layer, a pooling layer, a fully connected layer, and an output layer. The input layer contains processed audio data, with a size of 128*1. The input data is a one-dimensional vector representing time-series audio data. The network structure designed in this invention includes 6 convolutional blocks (each convolutional block contains one convolutional layer and one pooling layer), three fully connected layers, and one output layer. The convolutional layer in this invention contains 80 convolutional kernels of size 3*1 with a stride of 3. The pooling layer uses max pooling, with a pooling region size of 2*1. After the hidden features are extracted by the convolutional blocks, the element matrix is ​​flattened and fed into the fully connected layer. A Dropout layer is added in the fully connected layer to prevent overfitting during network training. The selected activation function is the ReLU function, the objective function is MSE, and the output layer consists of two neurons, where 0 indicates no and 1 indicates yes.

[0054] like Figure 1 As shown, the training process of the neural network in this invention is as follows: First, the sample data is divided into training data and test data, and the weights and biases are initialized. Then, the training data is input into the constructed neural network model. The weights and biases are continuously updated through forward propagation of information and backward propagation of error until the target function no longer decreases or decreases to the set threshold. When the training set and the test set perform in a consistent manner, the network training is completed.

[0055] Specifically, during neural network training: facial expressions and sounds of people of different ages and genders sneezing are recorded; blinking feature data is recorded using an infrared sensor; mouth feature data is recorded using an infrared camera; and the volume, audio, and duration of a sneeze are measured using a sound meter. A database is built based on this data, and the neural network is trained using data from this database. The facial expression and sound feature data of the samples are fed as inputs into the constructed 2D and 1D convolutional neural network training models, respectively. The 2D convolutional neural network's input is the person's facial expression output (whether the mouth is open or the eyes are closed), while the 1D convolutional neural network's output is whether there is a 'sneez' sound. The trained neural network is then used to identify whether the driver is sneezing. For a test sample, if both networks simultaneously output "open mouth," "closed eyes," and "sneez," then the sample indicates a sneeze, meaning the driver is sneezing, and the driver should be promptly alerted and the vehicle should be slowed down.

[0056] When the controller determines whether the driver has sneezed based on the driver's eye movements, mouth movements, and sound information, it calculates and analyzes these information and compares them with preset sneezing characteristics. Specifically, the controller's computer analyzes and compares the driver's eye characteristics transmitted by sensors with preset sneezing characteristics in the database, including the degree of eye closure, eye muscle movements, and duration of eye closure; it analyzes and compares the mouth characteristics transmitted by sensors with preset sneezing characteristics in the database, including the degree of mouth opening and closing and the duration of mouth opening and closing; and it analyzes and compares the sound characteristics transmitted by sensors with preset sneezing sound characteristics in the database, including the obvious changes in the frequency of the "achoo" sound and the instantaneous increase in loudness. If the driver's eye movements, mouth movements, and sound information all match the preset characteristics, the controller sends a control command to the actuator; if any one of them does not match, the controller does not send a control command to the actuator.

[0057] The controller's preset sneezing features, including driver eye movements, mouth movements, and vocal characteristics, are collected through big data analysis. This involves creating a sufficiently large sample with equal proportions of males, females, and different age groups. The system then collects the eye, mouth, and vocal characteristics of each individual sneezing, and image and audio processing software extracts these features. These features are then analyzed and integrated to create a preset database. This database undergoes multiple random tests to verify its accuracy. During random testing, any individual is randomly selected multiple times, and their sneezing characteristics are compared with the preset data to verify accuracy. This process continues until the data is sufficient to detect whether a person has sneezed; at this point, the preset database is considered the final preset database.

[0058] The working process of the alarm and control device for preventing sneezing from affecting driving safety according to the present invention includes the following steps:

[0059] (1) During driving, the sensor devices, namely image acquisition devices and audio acquisition devices, collect the driver's eye movements, mouth movements and sound information, and transmit the collected information to the controller.

[0060] (2) The controller is preset with the eye, mouth and sound characteristics of sneezing that have been collected and analyzed in a large number of samples. When the sensor sends information to the controller, the controller compares the collected image and sound information with the preset. If the three monitoring items (i.e. the eye, mouth and sound characteristics of sneezing) all match the characteristics of sneezing, the controller sends a command to the actuator.

[0061] (3) The actuator is connected to an audio playback device. After the actuator receives the instruction from the controller, the playback device plays the recorded voice to issue an alarm and automatically controls the throttle to slow down the vehicle but not to brake suddenly, thereby preventing a safety accident caused by the driver sneezing.

[0062] Example

[0063] Following the above technical solutions, such as Figures 1 to 2 As shown in the figure, this embodiment provides a sensing and alarm device to prevent sneezing from affecting driving safety. During driving, the sensing device determines whether the driver is about to sneeze, and if a sneeze is about to occur, an alarm is triggered in time.

[0064] The alarm device for preventing sneezing from affecting driving safety in this embodiment includes a sensor, a controller, and an actuator;

[0065] In this embodiment, preventing sneezing from affecting driving safety mainly includes four steps. In the first step, an image acquisition device uses an infrared high-definition camera to capture the driver's eye and mouth movements. In the second step, an audio acquisition device captures the driver's voice in real time. In the third step, the collected data is compared with standard data to determine if the driver has sneezed. In the fourth step, if the result of the third step is yes, a voice alarm is triggered.

[0066] Thus, the alarm device provided in this embodiment for preventing sneezing from affecting driving safety accurately determines whether the driver is about to sneeze by sequentially detecting and judging the driver's eye, mouth, and vocal states, and provides a timely voice alarm. Specifically, in the first step of using the image acquisition device, it is important to note that an infrared high-definition camera is used first, mainly because it can collect real-time information on the driver's eye and mouth states regardless of whether the vehicle is driving during the day or night. Secondly, regarding the preset eye movements when a person sneezes, three aspects are considered: the degree of eye closure, the duration of eye closure, and the characteristics of the muscles around the eyes. That is, if the degree of eye closure exceeds a preset value, the duration of eye closure exceeds a preset value, and the characteristics of the surrounding muscles are similar to the preset values, then the driver's eyes are judged to be in the eye state of sneezing. Regarding the preset mouth movements when a person sneezes, these include the degree of mouth opening and closing and the duration of mouth opening and closing. That is, if the degree of mouth opening and closing and the duration of mouth opening and closing exceed preset values, then the driver's mouth is judged to meet the mouth state of sneezing. In the second step, the audio equipment monitors the driver's voice in real time. The preset characteristics of a sneeze include its frequency and loudness. Because the "achoo" sound of a sneeze has a noticeable change in sound frequency and a sudden increase in loudness, a sneeze is considered to occur if the frequency and loudness are close to 90% similar to the preset values. The controller only determines that the driver is sneezing when all three characteristics are met. This is to improve the accuracy of the device and prevent false alarms that could disturb the driver and other occupants.

[0067] When determining the state of the eyes and mouth, the process begins with scanning using an infrared camera. The acquired image data is then processed, extracting feature data through image processing techniques such as ROI analysis, noise removal via filters, and nonlinear transformations. Once the feature data is extracted, it is compared with relevant parameters. For example, eye and mouth feature data extracted from images captured by a high-definition camera are compared with preset data. If the data falls within the preset range, it indicates the driver is sneezing. Of course, image acquisition devices are not limited to infrared camera scanning.

[0068] In determining the sound of a sneeze, the process involves first acquiring and transmitting audio data in real time using an audio acquisition device. Then, the acquired audio data is processed, for example, using a microphone. An audio analyzer is then used to extract and analyze the frequency and loudness of the sound. The obtained data is compared with preset data. If it falls within the preset range, the driver's sound is determined to be a sneeze.

[0069] For the actuator, the alarm voice should be pre-recorded and adjusted to an appropriate level, while the throttle control should be set to force release and briefly lock.

[0070] The above description of the disclosed embodiments enables those skilled in the art to make or use the invention. Various modifications to these embodiments will be readily apparent to those skilled in the art, and the general principles defined herein may be implemented in other embodiments without departing from the spirit or scope of the invention. Therefore, the invention is not to be limited to the embodiments shown herein, but is to be accorded the widest scope consistent with the principles and novel features disclosed herein.

Claims

1. A method for determining whether a driver has sneezed, characterized in that, The process includes the following: A neural network model is trained using a database to obtain a trained neural network model. The database includes sound feature data, eye feature data, and mouth feature data of a driver sneezing. The trained neural network model is used to process the real-time collected driver's eye feature data, mouth feature data, and voice feature data to obtain the feature data corresponding to the driver's eye feature, mouth feature, and voice feature respectively. Determine whether the driver sneezed based on the characteristic data corresponding to the driver's eye features, mouth features, and voice features; The feature data corresponding to the eye features includes eye image features and eye closure duration data; the feature data corresponding to the mouth features includes the degree of mouth opening; and the feature data corresponding to the voice features includes volume, frequency, and duration data. The process of determining whether a driver has sneezed based on the characteristic data corresponding to the driver's eye features, mouth features, and voice features includes: If the driver closes their eyes, the duration of eye closure reaches the preset duration of eye closure when sneezing, the degree of mouth opening reaches the degree of mouth opening when sneezing, and the sound emitted by the driver is the "ah-choo" sound when sneezing, then it is determined that the driver sneezed at this time; otherwise, it is determined that the driver did not sneeze. The data extraction process for the duration of eye closure includes: integrating and converting the collected duration of eye closure data into a one-dimensional vector of time data, and using the one-dimensional vector of time data as the duration of eye closure data; The process of extracting the degree of mouth opening includes: performing image transformation, image encoding compression, image enhancement and restoration, and image segmentation on the original image of the mouth to obtain two-dimensional image data of the mouth; using the two-dimensional image data of the mouth as mouth image features, and the mouth image features including the degree of mouth opening.

2. The method for determining whether a driver has sneezed according to claim 1, characterized in that, The process of extracting eye image features includes: The original images of the eyes are subjected to image transformation, image encoding compression, image enhancement and restoration, and image segmentation to obtain two-dimensional image data of the eyes. The two-dimensional image data of the eyes is then used as the features of the eye images.

3. The method for determining whether a driver has sneezed according to claim 1, characterized in that, The process of extracting the sound feature data includes: The collected audio is noise-reduced and compressed, and then the volume, frequency and duration of the audio are extracted. Finally, the extracted volume, frequency and duration are graded and quantized to obtain a one-dimensional vector, which is used as the sound feature data.

4. The method for determining whether a driver has sneezed according to claim 1, characterized in that, The process of using a trained neural network model to process real-time collected driver eye feature data, mouth feature data, and voice feature data to obtain the corresponding feature data for the driver's eye features, mouth features, and voice features includes: The trained neural network model is used to process eye image features to determine whether the driver's eyes are closed. The trained neural network model is used to process the data on the duration of eye closure to determine whether the duration of eye closure reaches the preset duration of eye closure when sneezing. The trained neural network model is used to process data including the degree of mouth opening to determine whether the degree of mouth opening is equal to the degree of mouth opening when the driver sneezes. The trained neural network model is used to process the sound feature data to determine whether the sound emitted by the driver is a sneezing sound.

5. A system for determining whether a driver has sneezed, characterized in that, The method for determining whether a driver has sneezed, as described in any one of claims 1-4, comprises: Eye feature data acquisition unit: used to record driver's eye movement information in real time and extract driver's eye feature data from the driver's eye movement information; Mouth feature data acquisition unit: used to record the driver's mouth movement information in real time and extract the driver's mouth feature data from the driver's mouth movement information; The sound feature data acquisition unit is used to collect the sound emitted by the driver in real time and extract sound feature data from the sound emitted by the driver. Data processing unit: Used to process real-time collected driver eye feature data, mouth feature data and voice feature data using a neural network model, to obtain feature data corresponding to the driver's eye features, mouth features and voice features respectively. Judgment module: used to determine whether the driver sneezes based on the feature data corresponding to the driver's eye features, mouth features, and voice features; The neural network model includes a two-dimensional convolutional neural network model and a one-dimensional convolutional neural network model; When determining whether the driver's eyes are closed, the two-dimensional convolutional neural network model is used to process the eye image features; When determining whether the duration of eye closure reaches the preset duration of eye closure when sneezing, the one-dimensional convolutional neural network model is used to process the eye closure duration data. When determining whether the degree of mouth opening is equal to the degree of mouth opening when a driver sneezes, the two-dimensional convolutional neural network model is used to process the data containing the degree of mouth opening. The trained neural network model is used to process the sound feature data to determine whether the sound emitted by the driver is the sneezing sound. The basic structure of the two-dimensional convolutional neural network includes an input layer, six convolutional blocks, three fully connected layers, and an output layer. The input layer has a data size of 224*224, 3 channels, and each element represents a grayscale value. Each convolutional block contains one convolutional layer and one pooling layer. Each convolutional layer contains 100 convolutional kernels of size 3*3 with a stride of 2. The pooling layer uses max pooling, with a pooling region of size 3*3. After the hidden features are extracted by the convolutional blocks, the element matrix is ​​flattened and fed into the fully connected layer. A Dropout layer is added to the fully connected layer, using ReLU as the activation function and MSE as the objective function. The output layer consists of two neurons, with 0 indicating no and 1 indicating yes. The basic structure of the one-dimensional convolutional neural network includes an input layer, six convolutional blocks, three fully connected layers, and an output layer. The input layer receives 128*1 data points, which are one-dimensional vectors. Each convolutional block contains a convolutional layer and a pooling layer. The convolutional layer has 80 kernels of size 3*1 with a stride of 3. The pooling layer uses max pooling with a pooling region of size 2*1. After the hidden features are extracted by the convolutional blocks, the element matrix is ​​flattened and fed into the fully connected layer. A Dropout layer is added to the fully connected layer, using ReLU as the activation function and MSE as the objective function. The output layer consists of two neurons, with 0 indicating no and 1 indicating yes.

Images