Information warning method and system based on brain waves

Abstract

The present invention proposes an information warning method and system based on brainwaves, which are applied to an intelligent driving cockpit. The information warning method includes: obtaining a brainwave data set of a first target; inputting the brainwave data set of the first target into a pre-trained brainwave model to determine the brainwave change trend of the first target; determining the concentration of the first target based on the brainwave change trend; determining that the concentration is lower than a preset concentration value, and obtaining the facial fatigue characteristics of the first target; determining the fatigue state based on the concentration and facial fatigue characteristics of the first target; determining that the fatigue state of the first target is an extreme fatigue state, and generating a first perception warning instruction; in response to the first perception warning instruction, providing a perception warning prompt to the first target. The present application uses multi-feature fusion judgment to find that the target population is in an extremely fatigued state and promptly issues a warning to improve the accuracy of fatigue warnings.

Description

Technical Field

[0001] The present invention relates to the technical field of fatigue early warning, and in particular to an information early warning method and system based on brain waves. Background Art

[0002] Among traffic accidents, accidents caused by fatigue driving account for a high proportion. With the popularization of intelligent driving systems, the monitoring and early warning systems for fatigue driving are becoming more and more mature. In related technologies, the detection methods for fatigue driving are relatively simple, with low detection accuracy, which can easily lead to misjudgment of fatigue driving; and the existing early warning prompts are poor in effect, which cannot provide drivers and passengers with a good driving or riding experience.

[0003] Based on this, it is necessary to propose an information warning method and system based on brain waves to solve the above technical problems. Summary of the invention

[0004] The main purpose of the present invention is to provide an information warning method and system based on brain waves, aiming to solve the technical problems of low fatigue detection accuracy and poor warning effect in the prior art.

[0005] To achieve the above object, the present invention proposes an information early warning method based on brain waves, the information early warning method comprising:

[0006] Acquire a brain wave data set of a first target, wherein the brain wave data set of the first target includes brain wave data of the first target within a preset time;

[0007] Inputting the EEG data set of the first target into a pre-trained EEG model to determine the EEG change trend of the first target;

[0008] Determining the concentration of the first target based on the brain wave change trend;

[0009] Determining that the concentration is lower than a preset concentration value, and acquiring facial fatigue characteristics of the first target;

[0010] Determining a fatigue state based on the concentration and facial fatigue characteristics of the first target, wherein the fatigue state includes mild fatigue and extreme fatigue;

[0011] Determining that the fatigue state of the first target is an extreme fatigue state, and generating a first perception warning instruction;

[0012] In response to the first perception warning instruction, a perception warning prompt is provided to the first target.

[0013] Preferably, after the step of determining the fatigue state based on the concentration and facial fatigue characteristics of the first target, the step further includes:

[0014] Determining that the fatigue state of the first target is a mild fatigue state, and acquiring a seat state of the second target, wherein the seat state includes an in-seat state and an empty seat state;

[0015] Determining that the seat is in a seated state, and acquiring a brain wave data set of a second target;

[0016] Inputting the brain wave data set of the second target into a pre-trained brain wave model to determine the brain wave change trend of the second target;

[0017] determining the concentration of the second target based on the brain wave change trend;

[0018] Determine that the concentration of the second target is higher than a preset value, and send a first prompt message; wherein the first prompt message is used to prompt the second target to directly warn the first target.

[0019] Preferably, after the step of acquiring the seat status of the second target, the step further includes:

[0020] Determining that the seat state is an empty seat state, and generating a first perception warning instruction;

[0021] In response to the first perception warning instruction, a perception warning prompt is provided to the first target.

[0022] Preferably, after the step of determining the concentration of the second target based on the brain wave change trend, the step further includes:

[0023] Determining that the concentration of the second target is lower than a preset value, and generating a first perception warning instruction;

[0024] In response to the first perception warning instruction, a perception warning prompt is provided to the first target.

[0025] Preferably, after the step of providing a perception warning prompt to the first target in response to the user perception warning instruction, the step further includes:

[0026] reacquiring the EEG data set of the first target after a preset time;

[0027] Inputting the EEG data set of the first target into a pre-trained EEG model to determine the EEG change trend of the first target;

[0028] Determining the concentration of the first target based on the brain wave change trend;

[0029] Determining that the concentration of the first target is lower than a preset value, and generating a second perception warning instruction;

[0030] In response to the second perception warning instruction, a perception warning prompt is provided to the second target.

[0031] Preferably, the facial fatigue feature includes an eye feature, and the step of acquiring the facial fatigue feature of the first target includes:

[0032] Acquire facial video stream data of the first target;

[0033] Extracting eye feature image frames in the facial video stream data;

[0034] The eye feature image frame is processed to form an eye feature line set, and the eye feature line set is used to represent the facial fatigue characteristics of the first target.

[0035] Preferably, the step of determining the fatigue state based on the concentration and facial fatigue characteristics of the first target includes:

[0036] Performing model matching on the eye feature line set to obtain an eye feature matching result;

[0037] Determining facial feature fatigue based on the eye feature matching result;

[0038] Obtain fatigue influence weights corresponding to brain wave concentration and facial feature fatigue;

[0039] The fatigue state of the first target is determined based on the fatigue impact weight, the concentration and the facial feature fatigue.

[0040] Preferably, the calculation formula of facial feature fatigue is expressed as:

[0041]

[0042] Among them, Z2 represents the fatigue degree of facial features, Z0 represents the reference value of facial feature fatigue, and s n It represents the unit matching value obtained after the nth eye feature pixel is matched with the model, N represents the total number of eye feature pixels in the eye feature line set, n∈(1,N].

[0043] Preferably, the step of determining the fatigue state of the first target based on the fatigue impact weight, concentration and facial feature fatigue comprises:

[0044] The fatigue quantification value P is calculated based on the fatigue influence weight W1, fatigue influence weight W2, concentration Z1 and facial feature fatigue Z2, and P, W1, W2, Z1, Z2 satisfy the expression: P = W2*Z2-W1*Z1, 0<W1<1, 0<W2<1, where W1 is the fatigue influence weight corresponding to the concentration, and W2 is the fatigue influence weight corresponding to the facial feature fatigue;

[0045] Determining that the fatigue quantization value P is greater than a first preset value and less than a second preset value, and determining that the fatigue state is mild fatigue;

[0046] It is determined that the fatigue quantization value P is greater than or equal to a second preset value, and the fatigue state is determined to be extreme fatigue.

[0047] Furthermore, to achieve the above-mentioned purpose, the present invention can also provide an information warning system, including a memory, a processor and a control program stored in the memory for implementing the information warning method based on brain waves, and the processor is used to execute the control program for implementing the information warning method based on brain waves to implement the steps of the information warning method based on brain waves as described above.

[0048] In the technical solution of the present invention, firstly, a brain wave data set of a first target is obtained; then, the brain wave data set of the first target is input into a pre-trained brain wave model to determine the brain wave change trend of the first target; then, the concentration of the first target is determined based on the brain wave change trend; then, it is determined that the concentration is lower than a preset concentration value, and the facial fatigue characteristics of the first target are obtained; then, the fatigue state is determined based on the concentration and facial fatigue characteristics of the first target; then, it is determined that the fatigue state of the first target is an extreme fatigue state, and a first perception warning instruction is generated; finally, in response to the first perception warning instruction, a perception warning prompt is given to the first target; that is, when the brain waves of the target population are detected and preliminarily judged to be in a low concentration state, the facial fatigue characteristics of the target population are obtained, and the final fatigue state is comprehensively judged by the concentration and facial fatigue characteristics. In this way, a warning is issued in time when the target population is found to be in an extremely fatigued state through multi-feature fusion judgment, so as to improve the accuracy of fatigue warning. BRIEF DESCRIPTION OF THE DRAWINGS

[0049] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on the structures shown in these drawings without paying creative work.

[0050] Figure 1 This is a schematic diagram of the structure of the intelligent driving cockpit of the present invention;

[0051] Figure 2 This is a schematic diagram of the structure of an intelligent car steering wheel in an intelligent driving cockpit of the present invention;

[0052] Figure 3 It is a flow chart of an embodiment of an information early warning method based on brain waves of the present invention;

[0053] Figure 4 It is a flow chart of another embodiment of the information warning method based on brain waves of the present invention;

[0054] Figure 5 It is a flow chart of another embodiment of the information early warning method based on brain waves of the present invention;

[0055] Figure 6 It is a flow chart of another embodiment of the information early warning method based on brain waves of the present invention;

[0056] Figure 7 It is a flow chart of another embodiment of the information early warning method based on brain waves of the present invention;

[0057] Figure 8 It is a flow chart of another embodiment of the information early warning method based on brain waves of the present invention;

[0058] Fig. 9 It is a flow chart of another embodiment of the information early warning method based on brain waves of the present invention;

[0059] Fig.10 It is a flow chart of another embodiment of the information early warning method based on brain waves of the present invention;

[0060] Fig.11 A schematic diagram of the structure of the hardware operating environment involved in an embodiment of the information early warning system of the present invention.

[0061] The realization of the purpose, functional features and advantages of the present invention will be further explained in conjunction with embodiments and with reference to the accompanying drawings. DETAILED DESCRIPTION

[0062] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0063] It should be noted that all directional indications in the embodiments of the present invention (such as up, down, left, right, front, back, etc.) are only used to explain the relative position relationship, movement status, etc. between the components under a certain specific posture (as shown in the accompanying drawings). If the specific posture changes, the directional indication will also change accordingly.

[0064] In addition, the descriptions of "first", "second", etc. in the present invention are only used for descriptive purposes and cannot be understood as indicating or implying their relative importance or implicitly indicating the number of the indicated technical features. Therefore, the features defined as "first" and "second" may explicitly or implicitly include at least one of the features. In addition, "and / or" in the full text includes three solutions. Taking A and / or B as an example, it includes technical solution A, technical solution B, and technical solution that satisfies both A and B. In addition, the technical solutions between the various embodiments can be combined with each other, but it must be based on the ability of ordinary technicians in this field to implement. When the combination of technical solutions is contradictory or cannot be implemented, it should be deemed that such combination of technical solutions does not exist and is not within the scope of protection required by the present invention.

[0065] The present application provides an information warning method based on brain waves, and applies the method to an intelligent driving cockpit. When a vehicle is driving on the road, it is inevitable that dangerous driving conditions such as deviation from the course may occur. When the vehicle is in an abnormal state, the driver is required to immediately correct the state to return the vehicle to a normal state and ensure the safety of the vehicle. It is understandable that whether the deviation can be corrected immediately requires the driver to have a high degree of concentration and to promptly discover the abnormal state of the vehicle. When the driver is in a fatigue state such as dozing off, poor spirits, and low concentration, the vehicle is very likely to be in danger. Therefore, when the driver is fatigued, it is necessary to provide user perception prompts to the vehicle driver and / or passengers to ensure that the driver is always awake to ensure the safe driving of the vehicle.

[0066] See attached Figure 1-2 There are many forms of smart cars, such as traditional fuel cars or pure electric new energy cars, etc. Smart car 100 includes a car body, a steering wheel 110 and a safety seat arranged in the car body. The safety seat includes a driver's seat 120 and a passenger seat 130. In this embodiment, when it is detected that the driver's fatigue level is high, different touches are used to remind the user to pay attention to safe driving.

[0067] The steering wheel 110 of the smart car 100 is provided with an electric touch prompt component for the driver to perceive, such as a metal electrode ring 111. The metal electrode ring 111 can be controlled by a controller to output an electric stimulation current (the current is within a safe range). The electric stimulation current includes at least two different stimulation intensities. The higher the driver's fatigue level, the greater the electric stimulation intensity. The electric stimulation current is used to electrically stimulate the driver to achieve a warning effect.

[0068] At the same time, the passenger seat is also equipped with a tactile prompt component for user perception. There are many forms of tactile prompts, such as electrode tactile or mechanical physical tactile. Taking mechanical physical tactile as an example, mechanical physical tactile includes at least two different forces of pushing, hammering or knocking to provide passengers with user-perceived quantitative prompts. Among them, the strength of the tactile prompt is used for the user to perceive the quantified degree of driver fatigue, that is, the greater the strength of the physical tactile prompt, the higher the driver's fatigue. After the passenger is physically touched, it indicates that the driver is fatigued. At this time, the passenger can directly warn the driver in the car to stay awake or suggest the driver to stop and rest as soon as possible, thereby indirectly reminding the driver by reminding the passengers to achieve a warning effect.

[0069] It should be noted that both the electrical stimulation warning prompt and the mechanical physical touch warning prompt will produce a certain tactile sensation on the user. Therefore, as long as the tactile warning prompt is produced, it will affect the user's driving or riding experience, and the greater the tactile intensity, the worse the user's driving or riding experience.

[0070] Brain waves are some spontaneous rhythmic neural electrical activities, with a frequency range of 1-30 times per second, which can be divided into four bands, namely δ (1-3Hz), θ (4-7Hz), α (8-13Hz), and β (14-30Hz). In addition, when awake and focusing on something, a Y wave with a higher frequency than the β wave can often be seen, with a frequency of 30-80Hz; and other normal brain waves with more special waveforms, such as the δ wave, can also appear during sleep. Therefore, the degree of concentration of the target object can be determined based on the trend of brain wave changes.

[0071] The following will mainly describe the specific steps of the pre-admission management method based on the Internet of Things security platform. It should be noted that although the logical order is shown in the flowchart, in some cases, the steps shown or described can be performed in an order different from that here.

[0072] See attached Figure 3 The brain wave-based information warning method comprises the following steps:

[0073] S100, obtaining a brain wave data set of a first target, where the brain wave data set of the first target includes brain wave data of the first target within a preset time;

[0074] There are many ways to obtain brain waves, such as wearing a brain wave collector on the head of the target population or setting a brain wave collector at a position corresponding to the back of the head on the seat, etc., which are not limited here. In addition, the brain wave data is a brain wave data set within a certain period of time, such as a brain wave data set from the start of the vehicle to the current time. It can be understood that in this embodiment, the first target is the driver.

[0075] S200, inputting the brain wave data set of the first target into a pre-trained brain wave model to determine the brain wave change trend of the first target;

[0076] The pre-trained brain wave model is a brain wave model obtained by a limited number of machine learning based on the actual fatigue state and the corresponding brain wave data.

[0077] S300, determining the concentration of the first target based on the brain wave change trend;

[0078] In this embodiment, when the brain wave changes from β wave to δ wave, it indicates that the concentration of the target group is gradually decreasing, and the concentration of the current first target can be determined according to the change amplitude of the β wave to δ wave and the initial concentration. The initial concentration is calculated as 100 (i.e. when the driver just starts driving). The faster the change, the lower the current concentration, and the current concentration is between 0-100, such as 10, 20, 30, 40, 50, 60, 70, 80, 90, etc.

[0079] Step S300 specifically includes:

[0080] S3001, obtaining a brain wave change trend, and generating a brain wave change curve according to the brain wave change trend;

[0081] S3002, searching and determining a corresponding brain wave change image in a preset brain wave change database according to the brain wave change curve;

[0082] S3003, according to the brain wave change image obtained by searching, searching from a preset function library to determine the brain wave change function corresponding to the brain wave change image;

[0083] S3004. Based on the initial concentration, the current concentration is calculated according to the brain wave change function.

[0084] S400, determining that the concentration is lower than a preset concentration value, and obtaining facial fatigue features of the first target;

[0085] After the driver has been driving for a period of time, his concentration may be reduced due to fatigue, or due to making phone calls or communicating with passengers. Therefore, when it is determined that the concentration is lower than the preset concentration value, it cannot be judged that the driver is in a fatigued state based on this, but it is necessary to continue to obtain the facial fatigue features of the first target, and comprehensively judge the final fatigue state based on the concentration and facial fatigue features.

[0086] S500, determining a fatigue state based on the concentration and facial fatigue characteristics of the first target, wherein the fatigue state includes mild fatigue and extreme fatigue;

[0087] There are many ways to determine the fatigue state based on the concentration and facial fatigue characteristics of the first target. In the following embodiments, the weights of the influence of concentration and facial fatigue characteristics on the actual fatigue state are taken as an example for detailed description.

[0088] S600, determining that the fatigue state of the first target is an extreme fatigue state, and generating a first perception warning instruction;

[0089] S700: In response to the first perception warning instruction, a perception warning prompt is issued to the first target;

[0090] In this embodiment, when it is determined that the fatigue state of the first target is an extremely fatigued state, the driver must be "awakened" immediately to ensure driving safety, and the first target is directly warned. Specifically, an electrical stimulation current is generated on the steering wheel to electrically stimulate the driver through the electrical stimulation current to achieve a warning effect, and at this time, a larger electrical stimulation intensity is used for warning prompts (which has a certain impact on the driver's driving experience).

[0091] This application first obtains the brain wave data set of the first target; then inputs the brain wave data set of the first target into the pre-trained brain wave model to determine the brain wave change trend of the first target; then determines the concentration of the first target based on the brain wave change trend; then determines that the concentration is lower than the preset concentration value, and obtains the facial fatigue characteristics of the first target; then determines the fatigue state based on the concentration and facial fatigue characteristics of the first target; then determines that the fatigue state of the first target is an extreme fatigue state, and generates a first perception warning instruction; finally, in response to the first perception warning instruction, the first target is given a perception warning prompt. That is, when the brain waves of the target population are detected and preliminarily judged to be in a low concentration state, the facial fatigue characteristics of the target population are obtained, and the final fatigue state is comprehensively judged by the concentration and facial fatigue characteristics. In this way, through multi-feature fusion judgment, it is found that the target population is in an extremely fatigued state and a warning is issued in time to improve the accuracy of fatigue warning.

[0092] In order to further improve the accuracy of fatigue warning and make the information warning provide a good driving experience for the driver and passengers, refer to the attached Figure 4 , after the step of determining the fatigue state based on the concentration and facial fatigue characteristics of the first target, the step further includes:

[0093] S800, determining that the fatigue state of the first target is a mild fatigue state, and obtaining a seat state of the second target, wherein the seat state includes an in-seat state and an empty seat state;

[0094] S900, determining that the seat is in the seated state, and acquiring an EEG data set of a second target;

[0095] S1000, inputting the brain wave data set of the second target into a pre-trained brain wave model to determine the brain wave change trend of the second target;

[0096] S1100, determining the concentration of the second target based on the brain wave change trend of the second target;

[0097] S1200: Determine that the concentration of the second target is higher than a preset value, and send a first prompt message; wherein the first prompt message is used to prompt the second target to directly warn the first target.

[0098] Specifically, the second target is the passenger group, and the passenger group can be the front passenger or the rear passenger. When it is detected that the first target, i.e., the driver, is in a state of mild fatigue, if the driver is directly given an electrical stimulation warning, the driver will have a poor driving experience. Therefore, the seat state of the second target, i.e., the seat state of the passenger seat, is first detected. When the seat state of the passenger seat is detected to be in place, it means that there is at least one passenger in the vehicle. At this time, the brain wave data set of the second target is obtained, and then the brain wave data set of the second target is input into the pre-trained brain wave model to determine the brain wave change trend of the second target to determine the concentration of the second target, i.e., the passenger; when the concentration of the second target, i.e., the passenger, is higher than the preset value, it means that the passenger is in a sober state at this time. At this time, the cockpit system can send a first prompt information, such as a voice prompt, through a prompt device. The first prompt information is used to prompt the second target to directly warn the first target. For example, the cockpit system voice broadcasts "The driver is slightly fatigued driving, please remind the driver to stay awake", so that the passenger can take further measures to remind the driver to stay awake, such as actively talking to the driver or reminding the driver to stop and rest.

[0099] It is understandable that when the fatigue state of the first target, i.e. the driver, is in a mild fatigue state and the passenger seat is in an empty state, it is impossible to directly or face-to-face warn the driver through the passenger. At this time, the driver can only be electrically stimulated by electrical stimulation current to achieve the warning effect to ensure driving safety. Figure 5 , after the step of obtaining the seat state of the second target, the step further includes:

[0100] S1300, determining that the seat state is an empty seat state, and generating a first perception warning instruction;

[0101] S1400. In response to the first perception warning instruction, provide a perception warning prompt for the first target.

[0102] Specifically, when the driver's fatigue state is a mild fatigue state and the passenger seat state is an empty state, since the driver still has a certain degree of mental consciousness, at this time, a smaller electrical stimulation intensity can be used for early warning to remind the driver. While ensuring the warning effect, the use of a smaller electrical stimulation intensity for early warning can greatly reduce the impact of the electrical stimulation warning on the driver's driving experience.

[0103] To further improve the accuracy of fatigue warning and ensure driving safety, refer to the attached Figure 6 , after the step of determining the concentration of the second target based on the brain wave change trend, the step further includes:

[0104] S1500, determining that the concentration of the second target is lower than a preset value, and generating a first perception warning instruction;

[0105] S1600. In response to the first perception warning instruction, provide a perception warning prompt for the first target.

[0106] Specifically, when it is detected that the first target, i.e., the driver, is in a state of mild fatigue, and the second target, i.e., the passenger, has a low degree of concentration, the method of directly warning the first target by sending a first prompt message to prompt the second target may take a long time, that is, the effectiveness is poor. For example, the passenger may not be awakened yet, and the vehicle may be in a dangerous state. In order not to disturb the passenger's rest or other behavior and to improve the passenger's riding experience, the system directly generates an electrical stimulation current to issue a warning, so as to timely warn the driver to stay awake and focused.

[0107] It is understandable that when the first target, i.e. the driver, is detected to be in a state of mild fatigue, and the second target, i.e. the passenger, is less focused, the system directly generates an electrical stimulation current for early warning. Although this method can improve the passenger's riding experience without disturbing the passenger's rest or other behaviors, the driver is still in an inattentive state after the electrical stimulation current is used to give an early warning, indicating that the driver may be sleeping or have a sudden illness. Therefore, in this case, in order to ensure driving safety and sacrifice the passenger's riding experience, the cockpit system issues a perception early warning prompt to the passenger to quickly remind the passenger that the driver is in the above-mentioned situation.

[0108] See attached Figure 7 , after the step of providing a perception warning prompt to the first target in response to the user perception warning instruction, the step further includes:

[0109] S1700, reacquiring the brain wave data set of the first target after a preset time;

[0110] S1800, inputting the brain wave data set of the first target into a pre-trained brain wave model to determine the brain wave change trend of the first target;

[0111] S1900, determining the concentration of the first target based on the brain wave change trend;

[0112] S2000, determining that the concentration of the first target is lower than a preset value, and generating a second perception warning instruction;

[0113] S2100. In response to the second perception warning instruction, provide a perception warning prompt for the second target.

[0114] Specifically, the preset time can be 10S or 30S or 1min. Taking 30S as an example, after the first target (driver) is perceived and warned for 30S, the concentration of the first target (driver) is detected again. When the concentration of the first target is still lower than the preset value, it means that the driver has not been "awakened" by the electrical stimulation current. At this time, the cockpit system generates a physical knocking tactile prompt to the passenger to quickly remind the passenger that the driver has not been "awakened", so as to wake up the driver in person through the driver.

[0115] See attached Figure 8 , the facial fatigue feature includes an eye feature, and the step of acquiring the facial fatigue feature of the first target includes:

[0116] S4100, obtaining facial video stream data of the first target;

[0117] S4200, extracting eye feature image frames in the facial video stream data;

[0118] S4300: Process the eye feature image frame to form an eye feature line set, and use the eye feature line set to represent facial fatigue features of the first target.

[0119] In this embodiment, the driver's facial video stream is first obtained, and then the eye feature image frames are extracted. There are many ways to obtain the driver's facial video stream, such as installing a monitoring camera in the cockpit, aiming the camera at the driver's face, and after obtaining the facial video stream, the facial video stream is cropped to obtain the eye video stream, and then the eye feature image frames within a preset time are extracted and processed to form an eye feature line set, such as extracting eye feature image frames within 30S, and processing all the eye feature image frames in series to form an eye feature line set.

[0120] See attached Fig. 9 The step of determining the fatigue state based on the concentration and facial fatigue characteristics of the first target includes:

[0121] S5100, performing model matching on the eye feature line set to obtain an eye feature matching result;

[0122] Among them, for the eye feature matching result, the eye feature matching result includes multiple unit matching values. Specifically, the eye feature line set includes multiple eye feature pixels. Each eye feature pixel is matched with a standard pixel in the model respectively, thereby obtaining multiple unit matching values.

[0123] S5200, determining the facial feature fatigue degree based on the eye feature matching result;

[0124]

[0125] Among them, Z2 represents the fatigue degree of facial features, Z0 represents the reference value of facial feature fatigue, and s n It represents the unit matching value obtained after the nth eye feature pixel is matched with the model, N represents the total number of eye feature pixels in the eye feature line set, n∈(1,N].

[0126] S5300, obtaining fatigue influence weights corresponding to brain wave concentration and facial feature fatigue;

[0127] S5400. Determine the fatigue state of the first target based on the fatigue impact weight, concentration and facial feature fatigue.

[0128] Specifically, after obtaining the eye feature line set, the eye feature line set is matched with a fatigue model to obtain an eye feature matching result. The eye feature matching result is represented by a numerical value of 10-100, such as the eye feature matching degree is 10, 20, 30, 40, 50, 60, 70, 80, 90, 100; the higher the matching degree, the greater the fatigue degree. In this embodiment, the numerical value of the eye feature matching degree represents the magnitude of the facial feature fatigue degree, that is, the magnitude of the facial feature fatigue degree is within the numerical range of 10-100.

[0129] See attached Fig.10 The step of determining the fatigue state of the first target based on the fatigue impact weight, concentration and facial feature fatigue includes:

[0130] S5410. Calculate the fatigue quantization value P based on the fatigue influence weight W1, the fatigue influence weight W2, the concentration Z1 and the facial feature fatigue Z2, and P, W1, W2, Z1, and Z2 satisfy the expression: P = W2*Z2-W1*Z1.

[0131] In this embodiment, 0<W1<1, 0<W2<1, wherein W1 is the fatigue influence weight corresponding to the concentration, and W2 is the fatigue influence weight corresponding to the facial feature fatigue.

[0132] S5420, determining that the fatigue quantization value P is greater than a first preset value and less than a second preset value, and determining that the fatigue state is mild fatigue;

[0133] S5430: Determine whether the fatigue quantization value P is greater than or equal to a second preset value, and determine that the fatigue state is extreme fatigue.

[0134] Specifically, in this embodiment, the fatigue influence weight corresponding to the concentration represented by brain waves is represented by W1, and the fatigue influence weight corresponding to the fatigue represented by facial features is represented by W2. The fatigue degree Z2 is positively correlated with the fatigue quantification value P, that is, the greater the fatigue degree, the greater the fatigue quantification value, and the more tired the driver is. The concentration Z1 is negatively correlated with the fatigue quantification value P, that is, the smaller the concentration Z1, the greater the fatigue quantification value, and the more tired the driver is; and the values ​​of W1 and W2 are stored in the memory.

[0135] When the driver's concentration decreases, it is most likely caused by fatigue. Therefore, this embodiment takes W1=0.6 and W2=0.4 as an example. When the calculated value of P=W2*Z2-W1*Z1 is greater than the first preset value and less than the second preset value, it indicates that the fatigue state is mild fatigue; when the calculated value of P=W2*Z2-W1*Z1 is greater than or equal to the second preset value, it indicates that the fatigue state is extreme fatigue; the specific first preset value and the second preset value can be determined according to the driver's age and driving experience, with 50 as the first preset value and 60 as the second preset value.

[0136] In addition, the present invention can also provide an information early warning system, such as Fig.11 As shown, the information warning system may include: a processor 1001, such as a CPU, a communication bus 1002, a user interface 1003, a network interface 1004, and a memory 1005. Among them, the communication bus 1002 is used to realize the connection and communication between these components. The user interface 1003 may include a display screen (Display), an input unit such as a keyboard (Keyboard), and the optional user interface 1003 may also include a standard wired interface and a wireless interface. The network interface 1004 may optionally include a standard wired interface and a wireless interface (such as a WI-FI interface). The memory 1005 may be a high-speed RAM memory, or a stable memory (non-volatile memory), such as a disk memory. The memory 1005 may also be a storage device independent of the aforementioned processor 1001.

[0137] Those skilled in the art will understand that Fig.11 The information warning system structure shown in the figure does not constitute a limitation on the information warning system, and may include more or fewer components than shown in the figure, or a combination of certain components, or a different arrangement of components.

[0138] like Fig.11 As shown, the memory 1005 as a storage medium may include an operating system, a network communication module, a user interface module, and a control program for implementing an information early warning method based on brain waves. Among them, the operating system is a program for managing and controlling pre-network devices and software resources, supporting the operation of the network communication module, the user interface module, the control program for implementing an information early warning method based on brain waves, and other programs or software; the network communication module is used to manage and control the network interface 1004; the user interface module is used to manage and control the user interface 1003.

[0139] exist Fig.11 In the hardware structure of the information early warning system shown in the figure, the network interface 1004 is mainly used to connect to the background server and communicate data with the background server; the user interface 1003 is mainly used to connect to the client (user end) and communicate data with the client; the processor 1001 can call the control program for implementing the information early warning method based on brain waves stored in the memory 1005, and perform the following steps:

[0140] Acquire a brain wave data set of a first target, wherein the brain wave data set of the first target includes brain wave data of the first target within a preset time;

[0141] Inputting the EEG data set of the first target into a pre-trained EEG model to determine the EEG change trend of the first target;

[0142] Determining the concentration of the first target based on the brain wave change trend;

[0143] Determining that the concentration is lower than a preset concentration value, and acquiring facial fatigue characteristics of the first target;

[0144] Determining a fatigue state based on the concentration and facial fatigue characteristics of the first target, wherein the fatigue state includes mild fatigue and extreme fatigue;

[0145] Determining that the fatigue state of the first target is an extreme fatigue state, and generating a first perception warning instruction;

[0146] In response to the first perception warning instruction, a perception warning prompt is provided to the first target.

[0147] In addition, the present invention can also provide a readable storage medium, on which a control program is stored, and when the control program is executed by a processor, the steps of the above-mentioned brain wave-based information early warning method are implemented:

[0148] Acquire a brain wave data set of a first target, wherein the brain wave data set of the first target includes brain wave data of the first target within a preset time;

[0149] Inputting the EEG data set of the first target into a pre-trained EEG model to determine the EEG change trend of the first target;

[0150] Determining the concentration of the first target based on the brain wave change trend;

[0151] Determining that the concentration is lower than a preset concentration value, and acquiring facial fatigue characteristics of the first target;

[0152] Determining a fatigue state based on the concentration and facial fatigue characteristics of the first target, wherein the fatigue state includes mild fatigue and extreme fatigue;

[0153] Determining that the fatigue state of the first target is an extreme fatigue state, and generating a first perception warning instruction;

[0154] In response to the first perception warning instruction, a perception warning prompt is provided to the first target.

[0155] The above description is only a preferred embodiment of the present invention, and does not limit the patent scope of the present invention. All equivalent structural changes made by using the contents of the present invention specification and drawings under the inventive concept of the present invention, or directly / indirectly applied in other related technical fields are included in the patent protection scope of the present invention.

Claims

1. An information warning method based on brain waves, applied to an intelligent driving cockpit, characterized in that: The information early warning method comprises: Acquire a brain wave data set of a first target, wherein the brain wave data set includes brain wave data of the first target within a preset time; Inputting the brain wave data set into a pre-trained brain wave model to determine the brain wave change trend of the first target; Determining the concentration of the first target based on the brain wave change trend; Determining that the concentration is lower than a preset concentration value, and acquiring facial fatigue characteristics of the first target; Determining a fatigue state based on the concentration and facial fatigue characteristics of the first target, wherein the fatigue state includes mild fatigue and extreme fatigue; Determining that the fatigue state of the first target is an extreme fatigue state, and generating a first perception warning instruction; In response to the first perception warning instruction, a perception warning prompt is provided to the first target.

2. The brain wave-based information warning method according to claim 1, characterized in that: After the step of determining the fatigue state based on the concentration and facial fatigue characteristics of the first target, the method further includes: Determining that the fatigue state of the first target is a mild fatigue state, and obtaining a seat state of the second target, wherein the seat state includes an in-seat state and an empty seat state; Determining that the seat is in a seated state, and acquiring a brain wave data set of a second target; Inputting the brain wave data set of the second target into a pre-trained brain wave model to determine the brain wave change trend of the second target; determining the concentration of the second target based on the brain wave change trend; Determine that the concentration of the second target is higher than a preset value, and send a first prompt message; wherein the first prompt message is used to prompt the second target to directly warn the first target.

3. The brain wave-based information warning method according to claim 2, characterized in that: After the step of acquiring the seat state of the second target, the following step is further included: Determining that the seat state is an empty seat state, and generating a first perception warning instruction; In response to the first perception warning instruction, a perception warning prompt is provided to the first target.

4. The brain wave-based information warning method according to claim 2, characterized in that: After the step of determining the concentration of the second target based on the brain wave change trend, the method further includes: Determining that the concentration of the second target is lower than a preset value, and generating a first perception warning instruction; In response to the first perception warning instruction, a perception warning prompt is provided to the first target.

5. The brain wave-based information warning method according to claim 4, characterized in that: After the step of providing a perception warning prompt to the first target in response to the user's perception warning instruction, the step further includes: reacquiring the EEG data set of the first target after a preset time; Inputting the EEG data set of the first target into a pre-trained EEG model to determine the EEG change trend of the first target; Determining the concentration of the first target based on the brain wave change trend; Determining that the concentration of the first target is lower than a preset value, and generating a second perception warning instruction; In response to the second perception warning instruction, a perception warning prompt is provided to the second target.

6. The brain wave-based information warning method according to claim 1, characterized in that: The facial fatigue feature includes an eye feature, and the step of acquiring the facial fatigue feature of the first target includes: Acquire facial video stream data of the first target; Extracting eye feature image frames in the facial video stream data; The eye feature image frame is processed to form an eye feature line set, and the eye feature line set is used to represent the facial fatigue characteristics of the first target.

7. The brain wave-based information warning method according to claim 6, characterized in that: The step of determining the fatigue state based on the concentration and facial fatigue characteristics of the first target includes: Performing model matching on the eye feature line set to obtain an eye feature matching result; Determining facial feature fatigue based on the eye feature matching result; Obtain fatigue influence weights corresponding to brain wave concentration and facial feature fatigue; The fatigue state of the first target is determined based on the fatigue impact weight, the concentration and the facial feature fatigue.

8. The brain wave-based information warning method according to claim 7, characterized in that: The calculation formula of facial feature fatigue is expressed as: Among them, Z2 represents the fatigue degree of facial features, Z0 represents the reference value of facial feature fatigue, S n It represents the unit matching value obtained after the nth eye feature pixel is matched with the model, N represents the total number of eye feature pixels in the eye feature line set, n∈(1,N].

9. The brain wave-based information warning method according to claim 8, characterized in that: The step of determining the fatigue state of the first target based on the fatigue impact weight, the concentration and the facial feature fatigue comprises: The fatigue quantification value P is calculated based on the fatigue influence weight W1, fatigue influence weight W2, concentration Z1 and facial feature fatigue Z2, and P, W1, W2, Z1, Z2 satisfy the expression: P = W2*Z2-W1*Z1, 0<W1<1, 0<W2<1, where W1 is the fatigue influence weight corresponding to the concentration, and W2 is the fatigue influence weight corresponding to the facial feature fatigue; Determining that the fatigue quantization value P is greater than a first preset value and less than a second preset value, and determining that the fatigue state is mild fatigue; It is determined that the fatigue quantization value P is greater than or equal to a second preset value, and the fatigue state is determined to be extreme fatigue.

10. An information early warning system, characterized in that: It includes a memory, a processor, and a control program stored in the memory for implementing the information early warning method based on brain waves, and the processor is used to execute the control program for implementing the information early warning method based on brain waves to implement the steps of the information early warning method based on brain waves as described in any one of claims 1 to 9.

Images