Automatic vehicle control method, system and equipment based on physiological state of driver

Abstract

The invention provides an automatic vehicle control method, system and equipment based on the physiological state of a driver, and the method comprises the steps: collecting the physiological data of the driver in a vehicle according to a preset time interval, and obtaining a first physiological data sequence; wherein the physiological data comprises at least one of heart rate, respiratory rate, eye blinking frequency, pupil size, facial expression, skin conductivity and body temperature; performing data optimization processing on the first physiological data sequence to obtain a second physiological data sequence; based on the second physiological data sequence, determining the probability that the driver is about to generate a physiological state; wherein the physiological state comprises at least one of sneezing, cough, fatigue, distraction and pain; and controlling the vehicle based on the probability of the physiological state that the driver is about to generate, so that the vehicle is controlled according to the physiological state of the driver, and the driving safety of the vehicle is improved.

Description

Technical Field

[0001] The present invention relates to the field of vehicle control technology, and in particular to a vehicle automatic control method, system and device based on the physiological state of a driver. Background Art

[0002] At present, vehicles have made significant progress in autonomous driving, and have achieved fully autonomous driving from simple assisted driving. However, during the autonomous driving process, the driver is still in the role of monitoring the vehicle and emergency intervention, especially in highly automated driving modes, the driver's physiological state is directly related to the overall driving safety. For example, when the vehicle is driving at high speed, the driver may temporarily lose control of the vehicle due to sudden physiological conditions (such as sneezing, coughing, drowsiness, etc.). If the above situation is not handled properly, it may cause serious traffic accidents.

[0003] Existing solutions can often only detect whether the driver is in a state of concentration, but are unable to effectively identify the driver's specific physiological abnormalities and take targeted response measures, posing a hidden danger to safe driving. Summary of the invention

[0004] The present invention provides a vehicle automatic control method, system and device based on the physiological state of a driver, so as to solve the technical problem in the prior art that the vehicle cannot be controlled according to the physiological state of the driver.

[0005] In one aspect, the present invention provides a vehicle automatic control method based on a driver's physiological state, comprising: Collecting physiological data of the driver in the vehicle at a preset time interval to obtain a first physiological data sequence; wherein the physiological data includes at least one of heart rate, breathing rate, eye blinking rate, pupil size, facial expression, skin conductivity and body temperature; Performing data optimization processing on the first physiological data sequence to obtain a second physiological data sequence; Determining, based on the second physiological data sequence, a probability of the driver's upcoming physiological state; wherein the physiological state includes at least one of sneezing, coughing, fatigue, distraction, and pain; The vehicle is controlled based on a probability of an upcoming physiological state of the driver.

[0006] According to a vehicle automatic control method based on the physiological state of a driver provided by the present invention, a piezoelectric sensor is embedded in the seat cushion of the driver's seat of the vehicle; The seat back of the driver's seat is embedded with a pressure sensor; An infrared camera is provided inside the vehicle; A camera is provided inside the vehicle; The steering wheel of the vehicle is integrated with a skin electrical sensor; The vehicle is provided with an infrared body temperature sensor; The step of collecting physiological data of the driver in the vehicle at a preset time interval includes: collecting the heart rate of the driver on the seat cushion based on the piezoelectric sensor; collecting the breathing frequency of the driver on the seat cushion based on the pressure sensor; Collecting the driver's eye blinking frequency and pupil size based on the infrared camera; Collecting the driver's facial expression based on the infrared camera; collecting the driver's skin conductivity based on the skin electrical sensor; The driver's body temperature is collected based on the infrared body temperature sensor.

[0007] According to a vehicle automatic control method based on the driver's physiological state provided by the present invention, the data optimization processing of the first physiological data sequence includes: Determining whether there is missing data in the first physiological data sequence; If yes, determine the previous data and the next data of the missing data; Determine an average value of the previous data and the next data, and use the average value to fill the missing data; Normalization processing is performed on the physiological data in the first physiological data sequence.

[0008] According to a vehicle automatic control method based on a driver's physiological state provided by the present invention, the facial expression is represented by one-hot encoding; The eye blinking frequency, the skin conductivity, the heart rate, the breathing rate, the pupil size and the body temperature are represented by real number coding.

[0009] According to a vehicle automatic control method based on a driver's physiological state provided by the present invention, the vehicle is controlled based on the probability of the driver's upcoming physiological state, including: When the probability of sneezing reaches a first threshold or the probability of coughing reaches a second threshold, an alarm is issued and the vehicle speed is automatically reduced by a first ratio; When the probability of fatigue reaches a third threshold, an alarm is issued and the vehicle speed is reduced by a second ratio; When the probability of fatigue reaches a third threshold and lasts for more than the first duration, the vehicle is completely taken over, the vehicle is automatically decelerated, and a parking place is sought for parking; When the probability of distraction reaches a fourth threshold, issuing an alarm prompt; When the probability of pain reaches a fifth threshold, the vehicle is completely taken over, the vehicle is automatically decelerated and a parking place is sought for parking, and a preset emergency contact is automatically contacted, and the physical condition of the driver is reported to the emergency contact; When the total probability value of the physiological state exceeds a sixth threshold, the vehicle is completely taken over, the vehicle is automatically decelerated, and a parking place is sought.

[0010] According to a vehicle automatic control method based on the physiological state of a driver provided by the present invention, after the vehicle is completely taken over, the method further comprises: Outputs a reminder message to inform the driver that the vehicle has entered autonomous driving mode and the steps to regain control of the vehicle.

[0011] According to a vehicle automatic control method based on a driver's physiological state provided by the present invention, the determining, based on the second physiological data sequence, the probability of the driver's upcoming physiological state comprises: Inputting the second physiological data sequence into a physiological state recognition model, and determining the probability of the driver's upcoming physiological state based on the physiological state recognition model; Wherein, the physiological state recognition model includes a first hidden layer, a second hidden layer, a first generator and a classifier; The first hidden layer includes N hidden units, each hidden unit inputs a corresponding sequence unit in the second physiological data sequence to obtain a hidden state sequence; wherein the hidden state sequence includes N hidden state features; The second hidden layer includes N attention units, each attention unit inputs a corresponding sequence unit in the hidden state sequence to obtain a weight feature sequence; wherein the weight feature sequence includes N weight features; The first generator is used to input the hidden state sequence and the weight feature sequence to obtain a final hidden state feature; The classifier is used to input the final hidden state features to obtain the probabilities of various physiological states.

[0012] According to a vehicle automatic control method based on a driver's physiological state provided by the present invention, before determining the probability of the driver's upcoming physiological state based on the second physiological data sequence, the method further includes: Obtaining training sample data of the physiological state recognition model based on a generative adversarial network model; Training the physiological state recognition model based on the training sample data; Wherein, the generative adversarial network model includes a second generator and a discriminator; The second generator inputs random noise or physiological data of the driver and outputs physiological state data samples; The input of the discriminator is the physiological state data sample and the real physiological state data, and the output is the probability value of the real physiological state data; The generative adversarial network model is trained by the following steps: Fixing the second generator and training the discriminator; Fixing the discriminator and training the second generator; The second generator and the discriminator are updated using a cross entropy function as a loss function, and the generated physiological state data are labeled to obtain training sample data of the physiological state recognition model.

[0013] On the other hand, the present invention also provides a vehicle automatic control system based on the physiological state of the driver, comprising: A collection module, used to collect physiological data of the driver in the vehicle at a preset time interval to obtain a first physiological data sequence; wherein the physiological data includes at least one of heart rate, breathing rate, eye blinking rate, pupil size, facial expression, skin conductivity and body temperature; an optimization module, configured to perform data optimization processing on the first physiological data sequence to obtain a second physiological data sequence; a state determination module, configured to determine, based on the second physiological data sequence, a probability of the driver's upcoming physiological state; wherein the physiological state includes at least one of sneezing, coughing, fatigue, distraction, and pain; The control module controls the vehicle based on the probability of the driver's upcoming physiological state.

[0014] On the other hand, the present invention also provides an electronic device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein when the processor executes the program, the vehicle automatic control method based on the driver's physiological state as described above is implemented.

[0015] The vehicle automatic control method, system and device based on the driver's physiological state provided by the present invention collect the physiological data of the driver in the vehicle at preset time intervals to obtain a first physiological data sequence, perform data optimization processing on the first physiological data sequence to obtain a second physiological data sequence, determine the probability of the driver's upcoming physiological state based on the second physiological data sequence, and control the vehicle based on the probability of the driver's upcoming physiological state, thereby effectively identifying the specific physiological abnormalities of the driver's physiological state and controlling the vehicle in a targeted manner, thereby ensuring the safety of vehicle driving and reducing the occurrence of traffic accidents. BRIEF DESCRIPTION OF THE DRAWINGS

[0016] In order to more clearly illustrate the technical solutions in the present invention or the prior art, the following briefly introduces the drawings required for use in the embodiments or the description of the prior art. Obviously, the drawings described below are some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0017] Figure 1 is a flow chart of a vehicle automatic control method based on a driver's physiological state provided by an embodiment of the present invention; Figure 2 is a schematic diagram of the structure of a vehicle automatic control system based on a driver's physiological state provided by an embodiment of the present invention; Figure 3 It is a schematic diagram of the structure of an electronic device provided by an embodiment of the present invention. DETAILED DESCRIPTION

[0018] In order to make the purpose, technical solution and advantages of the present invention clearer, the technical solution of the present invention will be clearly and completely described below in conjunction with the drawings of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0019] Figure 1 FIG. 1 is a flow chart of a vehicle automatic control method based on a driver's physiological state provided by an embodiment of the present invention. Figure 1 , the vehicle automatic control method based on the driver's physiological state may include the following steps.

[0020] 101. Collect physiological data of a driver in a vehicle at preset time intervals to obtain a first physiological data sequence; wherein the physiological data may include at least one of heart rate, breathing rate, eye blinking rate, pupil size, facial expression, skin conductivity and body temperature.

[0021] In this step, generally, the physiological data collected within the first preset time period T can be used as the first physiological data sequence. That is to say, a first physiological data sequence can be obtained every first preset time period T. The preset time interval t and the first preset time period T can be set, and are not specifically limited here. The first physiological data sequence can include N sequence units, and the sequence number of each sequence unit can be represented by n, that is, the nth sequence unit represents the physiological data at the nth time point. 1 is less than or equal to n is less than or equal to N. N=T / t.

[0022] 102. Perform data optimization processing on the first physiological data sequence to obtain a second physiological data sequence.

[0023] In this step, the data optimization processing of the first physiological data sequence may include: Determining whether there is missing data in the first physiological data sequence; If yes, determine the previous data and the next data of the missing data; An average value of the previous data and the next data is determined, and the missing data is filled with the average value.

[0024] Of course, performing data optimization processing on the first physiological data sequence may also include: Normalization processing is performed on the physiological data in the first physiological data sequence.

[0025] When there is missing data in the first physiological data sequence, the average value of the previous data and the next data of the missing data is determined, and the missing data is filled with the average value, which improves the completeness of the data in the first physiological data sequence and is conducive to the subsequent accurate identification of the driver's physiological state. Furthermore, normalizing the physiological data in the first physiological data sequence is conducive to eliminating the dimensional differences between different physiological data, ensuring that each physiological data is on the same scale, and making the analysis of physiological data easier. Generally, the first physiological data sequence can be normalized by maximum and minimum normalization.

[0026] 103. Based on the second physiological data sequence, determine the probability of the driver's upcoming physiological state; wherein the physiological state may include at least one of sneezing, coughing, fatigue, distraction and pain.

[0027] In this step, determining the probability of the driver's upcoming physiological state can also be understood as determining the possibility of the driver's upcoming physiological state. The greater the probability, the greater the possibility.

[0028] 104. Control the vehicle based on the probability of the driver's upcoming physiological state.

[0029] In this embodiment, the physiological data of the driver in the vehicle is collected at preset time intervals to obtain a first physiological data sequence, the first physiological data sequence is subjected to data optimization processing to obtain a second physiological data sequence, and the probability of the driver's upcoming physiological state is determined based on the second physiological data sequence. Based on the probability of the driver's upcoming physiological state, the vehicle is controlled, thereby achieving effective identification of specific physiological abnormalities in the driver's physiological state and targeted control of the vehicle, thereby ensuring the safety of vehicle driving and reducing the occurrence of traffic accidents.

[0030] In one embodiment of the present specification, a piezoelectric sensor is embedded in the seat cushion of the driver's seat of the vehicle; The seat back of the driver's seat is embedded with a pressure sensor; An infrared camera is provided inside the vehicle; A camera is provided inside the vehicle; The steering wheel of the vehicle is integrated with a skin electrical sensor; The vehicle is provided with an infrared body temperature sensor; The step of collecting physiological data of the driver in the vehicle at a preset time interval may include: collecting the heart rate of the driver on the seat cushion based on the piezoelectric sensor; collecting the breathing frequency of the driver on the seat cushion based on the pressure sensor; Collecting the driver's eye blinking frequency and pupil size based on the infrared camera; Collecting the driver's facial expression based on the infrared camera; collecting the driver's skin conductivity based on the skin electrical sensor; The driver's body temperature is collected based on the infrared body temperature sensor.

[0031] In this embodiment, the heart rate of the driver on the seat cushion is collected based on the piezoelectric sensor. Specifically, the piezoelectric sensor can sense the slight deformation of the seat caused by the driver's heartbeat to monitor and collect heart rate data. The piezoelectric sensor uses the characteristic of piezoelectric materials to generate electric charge when subjected to force to convert mechanical deformation into electrical signals. The driver's heartbeat causes slight deformation of the seat, and the piezoelectric sensor captures these deformations and converts them into heart rate data. In some other schemes, the driver's heart rate data can be obtained by connecting with the driver's sports bracelet, but there may be multiple drivers in the vehicle, and this way of obtaining heart rate is very inconvenient. The present invention collects the heart rate of the driver on the seat cushion based on the piezoelectric sensor, which breaks through the limitations of the above-mentioned problems, and uses a piezoelectric sensor, a device that has no direct relationship with heart rate measurement, to obtain the heart rate, breaking through the shackles of traditional thinking (using a device for measuring heart rate to obtain heart rate), which is very clever.

[0032] The breathing frequency of the driver on the seat cushion is collected based on the pressure sensor. Specifically, the breathing frequency data is monitored and collected by sensing the pressure changes caused by the expansion and contraction of the chest or abdomen when the driver breathes through the pressure sensor.

[0033] The driver's eye blinking frequency is collected based on the infrared camera. Specifically, the opening and closing state of the eyes is obtained by combining the infrared camera with a convolutional neural network, and then the eye blinking frequency is obtained.

[0034] The pupil size of the driver is collected based on the infrared camera, specifically, the pupil of the eye area at different positions at different time points is identified by combining the infrared camera with Hough transform and optical flow method, thereby obtaining the pupil size. Among them, Hough transform is used to identify the pupil edge, and optical flow method is used to track the change of the pupil position.

[0035] The driver's facial expression is collected based on the infrared camera. Specifically, the driver's facial expression is obtained by combining the camera with a convolutional neural network.

[0036] The skin conductivity of the driver is collected based on the skin electrical sensor. Specifically, the skin electrical sensor measures the change of the current passing through the driver's skin, thereby measuring the skin conductivity. An increase in the skin conductivity indicates that the driver is in a state of tension and anxiety. The formula for skin conductivity is: , SC represents skin conductivity, I represents the current passing through the skin, and V represents the voltage applied to the skin.

[0037] In one embodiment of the present specification, the facial expression is represented by one-hot encoding; The eye blinking frequency, the skin conductivity, the heart rate, the breathing rate, the pupil size and the body temperature are represented by real number coding.

[0038] In this embodiment, facial expressions may include drooping eyelids, sagging face, yawning, frowning, wrinkling nose, wide eyes and pale complexion. Facial expressions are represented by one-hot encoding. For example, a facial expression of frowning is represented by [0, 0, 0, 1, 0, 0, 0]. By one-hot encoding, facial expressions are converted into binary features to avoid comparison errors between numerical values. Eye blinking frequency is represented by real number encoding. For example, assuming 15 blinks per minute, the real number encoding is 15. By real number encoding, the relative size and relationship between numerical values ​​are retained, and the original numerical values ​​are used directly, reducing the computational overhead of encoding and decoding.

[0039] In one embodiment of the present specification, controlling the vehicle based on the probability of the driver's upcoming physiological state may include: When the probability of sneezing reaches a first threshold or the probability of coughing reaches a second threshold, an alarm is issued and the vehicle speed is automatically reduced by a first ratio; the alarm may include a slight sound reminder and a light visual reminder; When the probability of fatigue reaches a third threshold, an alarm is issued and the vehicle speed is reduced by a second ratio; the alarm here can be a relatively strong audio reminder; When the probability of fatigue reaches a third threshold and lasts for more than the first duration, the vehicle is completely taken over, the vehicle is automatically decelerated, and a parking place is sought for parking; When the probability of distraction reaches a fourth threshold, an alarm is issued; the alarm may include a continuous audio reminder and a seat vibration reminder; When the probability of pain reaches a fifth threshold, the vehicle is completely taken over, the vehicle is automatically decelerated and a parking place is sought for parking, and a preset emergency contact is automatically contacted, and the physical condition of the driver is reported to the emergency contact; When the total probability value of the physiological state exceeds a sixth threshold, the vehicle is completely taken over, the vehicle is automatically decelerated, and a parking place is sought.

[0040] In this embodiment, the first threshold value may be 0.3, the second threshold value may be 0.2, the third threshold value may be 0.4, the fourth threshold value may be 0.5, the fifth threshold value may be 0.6, and the sixth threshold value may be 0.8. The total probability value of the physiological state is the sum of the probability of sneezing, the probability of coughing, the probability of fatigue, the probability of distraction, and the probability of pain. The first ratio may be 10% to 20%, and the second ratio may be 10% to 20%. Assuming that the speed of the vehicle is 100 kilometers per hour, the first ratio is 10%, and the speed is reduced by the first ratio, and the speed is obtained = (speed-speed x first ratio) = 100-100x (10%) = 90. The above specific numerical values ​​are only examples and do not impose any limitations on the present invention. This embodiment effectively controls the vehicle according to the probability of the driver's upcoming physiological state, thereby ensuring driving safety.

[0041] In one embodiment of the present specification, after the complete takeover of the vehicle, the following steps may also be included: Outputs a reminder message to inform the driver that the vehicle has entered autonomous driving mode and the steps to regain control of the vehicle.

[0042] In this embodiment, after the vehicle is completely taken over, a reminder message is output to inform the driver that the vehicle has entered the automatic driving mode and the steps to regain control of the vehicle, so that the driver can regain control of the vehicle. For example, the specific method of gaining control of the vehicle can be obtained by issuing an audio broadcast.

[0043] In an embodiment of the present specification, determining the probability of the driver's upcoming physiological state based on the second physiological data sequence may include: Inputting the second physiological data sequence into a physiological state recognition model, and determining the probability of the driver's upcoming physiological state based on the physiological state recognition model; Wherein, the physiological state recognition model may include a first hidden layer, a second hidden layer, a first generator and a classifier; The first hidden layer may include N hidden units, each hidden unit inputs a corresponding sequence unit in the second physiological data sequence to obtain a hidden state sequence; wherein the hidden state sequence may include N hidden state features; The second hidden layer may include N attention units, each of which inputs a corresponding sequence unit in the hidden state sequence to obtain a weight feature sequence; wherein the weight feature sequence may include N weight features; The first generator is used to input the hidden state sequence and the weight feature sequence to obtain a final hidden state feature; The classifier is used to input the final hidden state features to obtain the probabilities of various physiological states.

[0044] In this embodiment, the hidden state feature is the physiological state, the weight feature is the weight of the physiological state, and the final hidden state feature is the physiological state carrying the weight.

[0045] Each hidden unit inputs the corresponding sequence unit in the second physiological data sequence, for example, the first hidden unit inputs the first sequence unit in the second physiological data sequence, and the nth hidden unit inputs the nth sequence unit in the second physiological data sequence. Each attention unit inputs the corresponding sequence unit in the hidden state sequence, for example, the first attention unit inputs the first sequence unit in the hidden state sequence, and the nth attention unit inputs the nth sequence unit in the hidden state sequence.

[0046] In some other embodiments of the present specification, the formula corresponding to the function of the nth hidden unit includes the following formulas (1) to (4).

[0047] Formula (1).

[0048] Formula (2).

[0049] Formula (3).

[0050] Formula (4).

[0051] Among them, n represents the sequence number, represents the nth sequence unit of the second physiological data sequence input by the nth hidden unit, and Represent the second candidate hidden features output by the nth and n-1th hidden units, respectively. Assign a value of 0, , and They represent the reset gate, update gate and first candidate hidden feature of the nth hidden unit of the first hidden layer, respectively. , and They respectively represent the first weight parameter, the second weight parameter and the bias parameter corresponding to the reset gate of the nth hidden unit of the first hidden layer, , and They respectively represent the first weight parameter, the second weight parameter and the bias parameter corresponding to the reset gate of the nth hidden unit of the first hidden layer, , and They respectively represent the first weight parameter, the second weight parameter and the bias parameter corresponding to the first candidate hidden feature of the nth hidden unit of the first hidden layer, represents point-by-point multiplication, sigmoid represents the sigmoid activation function, and tanh represents the tanh activation function.

[0052] In some other embodiments of the present specification, the formula of the second hidden layer includes the following formulas (5) to (9).

[0053] Formula (5).

[0054] Formula (6).

[0055] Formula (7).

[0056] Formula (8).

[0057] Formula (9).

[0058] in, represents the final hidden state features of the second hidden layer, represents the Nth intermediate hidden feature of the second hidden layer, represents the context vector feature of the Nth sequence unit in the second hidden layer, and represents the update gate and reset gate of the Nth hidden unit of the first hidden layer, and represents the second candidate hidden features output by the Nth and N-1th hidden units of the first hidden layer, , and represents the first weight parameter, the second weight parameter and the bias parameter of the Nth hidden unit of the first hidden layer, represents the first weight parameter corresponding to the reset gate of the Nth hidden unit of the first hidden layer, represents the Nth sequence unit of the second physiological data sequence input by the Nth hidden unit of the first hidden layer, represents the attention weight of the nth attention unit in the second hidden layer, represents the attention score of the Nth attention unit in the second hidden layer, represents the attention score outside the Nth attention unit of the second hidden layer, and The first weight parameter and bias parameter representing the attention score of the Nth attention unit in the second hidden layer, express and Vertical splicing.

[0059] In one embodiment of the present specification, before determining the probability of the driver's upcoming physiological state based on the second physiological data sequence, the method may further include: Obtaining training sample data of the physiological state recognition model based on a generative adversarial network model; Training the physiological state recognition model based on the training sample data; Wherein, the generative adversarial network model may include a second generator and a discriminator; The second generator inputs random noise or physiological data of the driver and outputs physiological state data samples; The input of the discriminator is a physiological state data sample and real physiological state data, and the output is a probability value of the real physiological state data.

[0060] In one embodiment of the present specification, the generative adversarial network model is trained by the following steps: Fixing the second generator and training the discriminator; Fixing the discriminator and training the second generator; The second generator and the discriminator are updated using a cross entropy function as a loss function, and the generated physiological state data are labeled to obtain training sample data of the physiological state recognition model.

[0061] In this embodiment, alternating training is performed: the generator is fixed to train the discriminator, and then the discriminator is fixed to train the generator. This alternating training method helps to update the model parameters stably. Cross entropy loss: Using the cross entropy function as the loss function can more effectively measure the difference between the generated data and the real data, and accelerate the convergence of the model.

[0062] Based on the same general inventive concept, the present invention also protects a vehicle automatic control system based on the driver's physiological state, such as Figure 2As shown. The vehicle automatic control system based on the driver's physiological state provided by the present invention is described below. The vehicle automatic control system based on the driver's physiological state described below and the vehicle automatic control method based on the driver's physiological state described above can be referred to each other. The vehicle automatic control system may include a collection module 201, an optimization module 202, a state determination module 203 and a control module 204.

[0063] The acquisition module 201 is used to collect physiological data of the driver in the vehicle at preset time intervals to obtain a first physiological data sequence; wherein the physiological data may include at least one of heart rate, breathing rate, eye blinking frequency, pupil size, facial expression, skin conductivity and body temperature.

[0064] The optimization module 202 is used to perform data optimization processing on the first physiological data sequence to obtain a second physiological data sequence.

[0065] The state determination module 203 is used to determine the probability of the driver's upcoming physiological state based on the second physiological data sequence; wherein the physiological state may include at least one of sneezing, coughing, fatigue, distraction and pain.

[0066] The control module 204 controls the vehicle based on the probability of the driver's upcoming physiological state.

[0067] Figure 3 It is a schematic diagram of the structure of an electronic device provided by an embodiment of the present invention.

[0068] like Figure 3 As shown, the electronic device may include: a processor 310, a communication interface 320, a memory 330 and a communication bus 340, wherein the processor 310, the communication interface 320 and the memory 330 communicate with each other through the communication bus 340. The processor 310 may call the logic instructions in the memory 330 to execute the vehicle automatic control method based on the physiological state of the driver.

[0069] In addition, the logic instructions in the above-mentioned memory 330 can be implemented in the form of a software functional unit and can be stored in a computer-readable storage medium when it is sold or used as an independent product. Based on such an understanding, the technical solution of the present invention, in essence, or the part that contributes to the prior art or the part of the technical solution, can be embodied in the form of a software product, and the computer software product is stored in a storage medium, including a number of instructions for a computer device (which can be a personal computer, a server, or a network device, etc.) to perform all or part of the steps of the method described in each embodiment of the present invention. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (ROM, Read-Only Memory), random access memory (RAM, Random Access Memory), disk or optical disk, etc. Various media that can store program codes.

[0070] On the other hand, the present invention also provides a computer program product, which includes a computer program. The computer program can be stored on a non-transitory computer-readable storage medium. When the computer program is executed by a processor, the computer can execute the vehicle automatic control method based on the driver's physiological state provided by the above-mentioned methods.

[0071] On the other hand, the present invention also provides a non-transitory computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, is implemented to execute the vehicle automatic control method based on the driver's physiological state provided by the above-mentioned methods.

[0072] The device embodiments described above are merely illustrative, wherein the units described as separate components may or may not be physically separated, and the components displayed as units may or may not be physical units, that is, they may be located in one place, or they may be distributed on multiple network units. Some or all of the modules may be selected according to actual needs to achieve the purpose of the scheme of this embodiment. Ordinary technicians in this field can understand and implement it without paying creative labor.

[0073] Through the description of the above implementation methods, those skilled in the art can clearly understand that each implementation method can be implemented by means of software plus a necessary general hardware platform, and of course, can also be implemented by hardware. Based on this understanding, the above technical solution is essentially or the part that contributes to the prior art can be embodied in the form of a software product, and the computer software product can be stored in a computer-readable storage medium, such as ROM / RAM, a disk, an optical disk, etc., including a number of instructions for a computer device (which can be a personal computer, a server, or a network device, etc.) to execute the methods described in each embodiment or some parts of the embodiments.

[0074] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit it. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or make equivalent replacements for some of the technical features therein. However, these modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. A vehicle automatic control method based on the driver's physiological state, characterized in that: include: Collecting physiological data of the driver in the vehicle at a preset time interval to obtain a first physiological data sequence; wherein the physiological data includes at least one of heart rate, breathing rate, eye blinking rate, pupil size, facial expression, skin conductivity and body temperature; Performing data optimization processing on the first physiological data sequence to obtain a second physiological data sequence; Determining, based on the second physiological data sequence, a probability of the driver's upcoming physiological state; wherein the physiological state includes at least one of sneezing, coughing, fatigue, distraction, and pain; The vehicle is controlled based on a probability of an upcoming physiological state of the driver.

2. The vehicle automatic control method based on the driver's physiological state according to claim 1 is characterized in that: The seat cushion of the driver's seat of the vehicle is embedded with a piezoelectric sensor; The seat back of the driver's seat is embedded with a pressure sensor; An infrared camera is arranged inside the vehicle; A camera is provided inside the vehicle; The steering wheel of the vehicle is integrated with a skin electrical sensor; The vehicle is provided with an infrared body temperature sensor; The step of collecting physiological data of the driver in the vehicle at a preset time interval includes: collecting the heart rate of the driver on the seat cushion based on the piezoelectric sensor; collecting the breathing frequency of the driver on the seat cushion based on the pressure sensor; Collecting the driver's eye blinking frequency and pupil size based on the infrared camera; Collecting the driver's facial expression based on the infrared camera; collecting the driver's skin conductivity based on the skin electrical sensor; The driver's body temperature is collected based on the infrared body temperature sensor.

3. The vehicle automatic control method based on the driver's physiological state according to claim 1, characterized in that: The performing data optimization processing on the first physiological data sequence includes: Determining whether there is missing data in the first physiological data sequence; If yes, determine the previous data and the next data of the missing data; Determine an average value of the previous data and the next data, and use the average value to fill the missing data; Normalization processing is performed on the physiological data in the first physiological data sequence.

4. The vehicle automatic control method based on the driver's physiological state according to claim 1, characterized in that: The facial expression is represented by one-hot encoding; The eye blinking frequency, the skin conductivity, the heart rate, the breathing rate, the pupil size and the body temperature are represented by real number coding.

5. The vehicle automatic control method based on the driver's physiological state according to claim 1, characterized in that: The controlling the vehicle based on the probability of the driver's upcoming physiological state comprises: When the probability of sneezing reaches a first threshold or the probability of coughing reaches a second threshold, an alarm is issued and the vehicle speed is automatically reduced by a first ratio; When the probability of fatigue reaches a third threshold, an alarm is issued and the vehicle speed is reduced by a second ratio; When the probability of fatigue reaches a third threshold and lasts for more than the first duration, the vehicle is completely taken over, the vehicle is automatically decelerated, and a parking place is sought for parking; When the probability of distraction reaches a fourth threshold, issuing an alarm prompt; When the probability of pain reaches a fifth threshold, the vehicle is completely taken over, the vehicle is automatically decelerated and a parking place is sought for parking, and a preset emergency contact is automatically contacted, and the physical condition of the driver is reported to the emergency contact; When the total probability value of the physiological state exceeds a sixth threshold, the vehicle is completely taken over, the vehicle is automatically decelerated, and a parking place is sought.

6. The vehicle automatic control method based on the driver's physiological state according to claim 5, characterized in that: After the complete takeover of the vehicle, the following steps are also included: Outputs a reminder message to inform the driver that the vehicle has entered autonomous driving mode and the steps to regain control of the vehicle.

7. The vehicle automatic control method based on the driver's physiological state according to claim 1, characterized in that: The determining, based on the second physiological data sequence, the probability of the driver's upcoming physiological state comprises: Inputting the second physiological data sequence into a physiological state recognition model, and determining the probability of the driver's upcoming physiological state based on the physiological state recognition model; Wherein, the physiological state recognition model includes a first hidden layer, a second hidden layer, a first generator and a classifier; The first hidden layer includes N hidden units, each hidden unit inputs a corresponding sequence unit in the second physiological data sequence to obtain a hidden state sequence; wherein the hidden state sequence includes N hidden state features; The second hidden layer includes N attention units, each attention unit inputs a corresponding sequence unit in the hidden state sequence to obtain a weight feature sequence; wherein the weight feature sequence includes N weight features; The first generator is used to input the hidden state sequence and the weight feature sequence to obtain a final hidden state feature; The classifier is used to input the final hidden state features to obtain the probabilities of various physiological states.

8. The vehicle automatic control method based on the driver's physiological state according to claim 7 is characterized in that: Before determining the probability of the driver's upcoming physiological state based on the second physiological data sequence, the method further includes: Obtaining training sample data of the physiological state recognition model based on a generative adversarial network model; Training the physiological state recognition model based on the training sample data; Wherein, the generative adversarial network model includes a second generator and a discriminator; The second generator inputs random noise or physiological data of the driver and outputs physiological state data samples; The input of the discriminator is the physiological state data sample and the real physiological state data, and the output is the probability value of the real physiological state data; The generative adversarial network model is trained by the following steps: Fixing the second generator and training the discriminator; Fixing the discriminator and training the second generator; The second generator and the discriminator are updated using a cross entropy function as a loss function, and the generated physiological state data are labeled to obtain training sample data of the physiological state recognition model.

9. A vehicle automatic control system based on the driver's physiological state, characterized in that: include: A collection module, used to collect physiological data of the driver in the vehicle at a preset time interval to obtain a first physiological data sequence; wherein the physiological data includes at least one of heart rate, breathing rate, eye blinking rate, pupil size, facial expression, skin conductivity and body temperature; an optimization module, configured to perform data optimization processing on the first physiological data sequence to obtain a second physiological data sequence; a state determination module, configured to determine, based on the second physiological data sequence, a probability of the driver's upcoming physiological state; wherein the physiological state includes at least one of sneezing, coughing, fatigue, distraction, and pain; The control module controls the vehicle based on the probability of the driver's upcoming physiological state.

10. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that: When the processor executes the program, the vehicle automatic control method based on the driver's physiological state as described in any one of claims 1 to 8 is implemented.

Images