Driver state monitoring and intervention method in plateau environment

Abstract

The invention belongs to the technical field of driving monitoring, and discloses a driver state monitoring and intervention method in a plateau environment, which comprises the following steps: acquiring plateau environment data and driving state data of a driver; transmitting the plateau environment data and the driving state data to a cloud computing node for multi-source data fusion processing to obtain corresponding plateau environment fusion data and driving state fusion data, based on the plateau environment fusion data and the driving state fusion data, constructing a function relational expression between the plateau environment change and the driving state; and executing real-time risk prediction of the to-be-monitored driver based on the constructed function relation to obtain a driving risk prediction result, and intervening the driving state of the to-be-monitored driver based on the driving risk prediction result. According to the technical scheme, the driving safety of a driver in the plateau area can be improved, and a reference basis for driving in the plateau environment can be provided for the driver without installation of detection equipment.

Description

Technical Field

[0001] The invention belongs to the technical field of driving monitoring, and in particular relates to a driver status monitoring and intervention method in a plateau environment. Background Art

[0002] When driving at high altitudes, drivers face multiple challenges brought by the complex terrain and natural environment of the plateau. These challenges include not only rugged roads, steep slopes and unpredictable weather conditions, but also the hypoxic environment unique to the plateau. In an oxygen-deficient environment, the human body will experience a series of physiological reactions, such as increased heart rate, increased blood pressure, and increased respiratory rate. These physiological changes will further affect the driver's psychological state and behavioral performance.

[0003] From a physiological perspective, plateau hypoxia can cause drivers to have reduced physical function, such as reduced physical endurance and slower reaction speed. These physiological changes will directly affect the driver's driving ability and increase driving risks. From a psychological perspective, the plateau environment will also have a significant impact on the driver's mental state. For example, long-term driving and hypoxic environments may trigger negative emotions such as anxiety and irritability in drivers, reducing their attention and judgment. In addition, the loneliness and monotony of the plateau may also aggravate the driver's psychological fatigue and further affect their driving performance. From a behavioral perspective, the complex road conditions and hypoxic environment of the plateau may cause drivers to make unsafe decisions and operations during driving. For example, in order to cope with the discomfort caused by hypoxia in the plateau, drivers may increase the number and time of rest, which may lead to longer driving time and disrupted driving plans. At the same time, hypoxia may also affect the driver's judgment, making it impossible for them to respond correctly in an emergency.

[0004] In summary, changes in the plateau environment have a significant impact on the driver's physiology, psychology and behavior. Therefore, it is particularly important to study the impact of the plateau environment on drivers and develop methods that can timely monitor and intervene in the driver's "physiological-psychological-behavioral" changes. Summary of the invention

[0005] The purpose of the present invention is to provide a driver status monitoring and intervention method in a plateau environment to solve the problems existing in the above-mentioned prior art.

[0006] To achieve the above object, the present invention provides a driver status monitoring and intervention method in a plateau environment, comprising:

[0007] Acquire plateau environment data and driver's driving status data;

[0008] The plateau environment data and the driving state data are transmitted to a cloud computing node for multi-source data fusion processing to obtain corresponding plateau environment fusion data and driving state fusion data, and a functional relationship between plateau environment changes and driving states is constructed based on the plateau environment fusion data and the driving state fusion data;

[0009] Based on the constructed functional relationship, real-time risk prediction of the driver to be monitored is performed to obtain a driving risk prediction result, and intervention is performed on the driving state of the driver to be monitored based on the driving risk prediction result.

[0010] Optionally, the plateau environmental data includes temperature and humidity data, air pressure data, oxygen content data, and wind speed and direction data.

[0011] Optionally, the driving status data includes the driver's physiological information data, psychological information data and driving behavior data.

[0012] Optionally, the process of obtaining the psychological information data specifically includes:

[0013] Acquiring facial image data of the driver;

[0014] The facial image data is input into a pre-trained expression recognition model to perform facial expression analysis to obtain emotional state data, and the emotional state data is uploaded to a cloud computing node as psychological information data; wherein the expression recognition model is constructed based on an MVCNN network.

[0015] Optionally, the process of acquiring the driving behavior data specifically includes:

[0016] The speeding and emergency stop conditions of the vehicle are detected based on the Hall-type wheel speed sensor, and the lane change conditions of the vehicle are detected using the laser radar. The detected speeding, emergency stop and lane change conditions are uploaded to the cloud computing node as driving behavior data.

[0017] Optionally, before transmitting the plateau environment data and the driving status data to the cloud computing node for multi-source data fusion processing, the plateau environment data is also included in data cleaning and format conversion to obtain pre-processed plateau environment data, and the pre-processed plateau environment data is transmitted to the cloud computing node through a microprocessor for multi-source data fusion processing.

[0018] Optionally, transmitting the plateau environment data and the driving status data to a cloud computing node for multi-source data fusion processing specifically includes:

[0019] The plateau environment data and the driving status data are respectively input into a multi-source data fusion module for data fusion to obtain corresponding plateau environment fusion data and driving status fusion data; wherein the multi-source data fusion module includes a BP neural network prediction module and a fuzzy logic control decision module set in parallel.

[0020] Optionally, the processing process of the multi-source data fusion module specifically includes:

[0021] The data to be input is input into the BP neural network prediction module and the fuzzy logic control decision module for data fusion. When the output results of the BP neural network prediction module and the fuzzy logic control decision module are consistent, the optimal fusion data is output; when the output results of the BP neural network prediction module and the fuzzy logic control decision module are inconsistent, the output result of the BP neural network prediction module is used as the main decision, and the output result of the fuzzy logic control decision module is used as the auxiliary decision, and the optimal fusion data is determined based on the main decision and the auxiliary decision.

[0022] Optionally, constructing a functional relationship between the plateau environment change and the driving state specifically includes:

[0023] Polynomial fitting is performed on the plateau environment fusion data and the driving state fusion data to obtain a functional relationship between the plateau environment change and the driving state. The specific calculation formula is:

[0024]

[0025] In the formula, M is the plateau environment fusion data, N is the driving state fusion data, and a n is the weight, x i (i=1,2,3…n) represents the multidimensional vector after the fusion of plateau environmental data, that is, M=(x 1 , x 2 , x 3 …x n ), y i (i=1,2,3…n) represents the multidimensional vector after the driver’s driving status data is fused, that is, N=(y 1 ,y 2 ,y 3 …y n ), N = f(M) represents the functional relationship between plateau environment changes and driving conditions.

[0026] Optionally, intervening in the driving state of the driver to be monitored based on the driving risk prediction result specifically includes:

[0027] Early warning intervention of the driver's physiological state to be monitored:

[0028] Based on Bayesian theorem, the driver's initial physiological state is taken as a priori probability, and the physiological state of the driver to be monitored when driving in a plateau environment is taken as a conditional probability. The posterior probability is calculated based on the priori probability and the conditional probability. When the posterior probability is greater than a preset threshold, an early warning prompt is given to the driver.

[0029] Intervene the negative emotions of the monitored driver:

[0030] When the monitored driver shows negative emotions, the driver's emotions can be alleviated by playing music and adjusting the color of the lights in the car.

[0031] The technical effects of the present invention are:

[0032] The present invention can not only instantly evaluate and intelligently intervene in the driver's status and ensure driving safety by comprehensively collecting complex parameters of the plateau environment and the driver's real-time driving status information, but also accurately predict the driving area of ​​the vehicle without equipment assistance in the plateau environment by constructing a refined mathematical model and simulation function, thereby comprehensively improving the safety and efficiency of driving on plateau roads. The present invention can not only improve the driving safety of drivers in plateau areas, but also provide a reference for driving in plateau environments for drivers who do not have detection equipment installed, helping them judge the plateau environment limit values ​​they can accept, so as to make wise driving decisions. BRIEF DESCRIPTION OF THE DRAWINGS

[0033] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0034] The drawings constituting a part of the present application are used to provide a further understanding of the present application. The illustrative embodiments and descriptions of the present application are used to explain the present application and do not constitute an improper limitation on the present application. In the drawings:

[0035] Figure 1 is a flow chart of a driver's "physiological-psychological-behavioral" state monitoring and intervention method in a plateau environment in an embodiment of the present invention;

[0036] Figure 2 is a distribution diagram of the external vehicle sensors in an embodiment of the present invention;

[0037] Figure 3 A schematic diagram of driver driving status collection in an embodiment of the present invention;

[0038] Figure 4 A plateau environment data generation diagram in an embodiment of the present invention;

[0039] Figure 5 A driving state data generation diagram in an embodiment of the present invention;

[0040] Figure 6 is a diagram of a driver emotion recognition network structure in an embodiment of the present invention;

[0041] Figure 7 is a flow chart of physiological state intervention of a device system in an embodiment of the present invention;

[0042] Figure 8 is a flow chart of psychological state intervention in a device system in an embodiment of the present invention;

[0043] Fig. 9 is a flowchart of driving behavior intervention with a device system in an embodiment of the present invention;

[0044] Fig.10 It is a schematic diagram of the equipment-free prediction model fitting training in an embodiment of the present invention. DETAILED DESCRIPTION

[0045] Various exemplary embodiments of the present invention will now be described in detail. This detailed description should not be considered as limiting the present invention, but should be understood as a more detailed description of certain aspects, features, and embodiments of the present invention.

[0046] It should be understood that the terms described in the present invention are only for describing special embodiments and are not intended to limit the present invention. In addition, for the numerical range in the present invention, it should be understood that each intermediate value between the upper and lower limits of the scope is also specifically disclosed. Each smaller range between the intermediate value in any stated value or stated range and any other stated value or intermediate value in the described range is also included in the present invention. The upper and lower limits of these smaller ranges can be independently included or excluded in the scope.

[0047] Unless otherwise indicated, all technical and scientific terms used herein have the same meanings as those of ordinary skill in the art generally understood by the present invention. Although the present invention describes only preferred methods, any method similar or equivalent to that described herein may also be used in the implementation or testing of the present invention. All documents mentioned in this specification are incorporated by reference to disclose and describe the methods related to the documents. In the event of a conflict with any incorporated document, the content of this specification shall prevail.

[0048] It will be apparent to those skilled in the art that various modifications and variations may be made to the specific embodiments of the present invention description without departing from the scope or spirit of the present invention. Other embodiments derived from the present invention description will be apparent to those skilled in the art. The present application description and examples are exemplary only.

[0049] The words “include,” “including,” “have,” “contain,” etc. used in this article are open-ended terms, meaning including but not limited to.

[0050] It should be noted that, in the absence of conflict, the embodiments and features in the embodiments of the present application can be combined with each other. The present application will be described in detail below with reference to the accompanying drawings and in combination with the embodiments.

[0051] Embodiment 1

[0052] like Figure 1 - Fig.10 As shown, this embodiment provides a driver status monitoring and intervention method in a plateau environment, including: obtaining plateau environment data and obtaining the driver's driving status data; transmitting the plateau environment data and the driving status data to a cloud computing node for multi-source data fusion processing to obtain corresponding plateau environment fusion data and driving status fusion data, and constructing a functional relationship between plateau environment changes and driving status based on the plateau environment fusion data and the driving status fusion data; performing real-time risk prediction of the driver to be monitored based on the constructed functional relationship to obtain a driving risk prediction result, and intervening in the driving status of the driver to be monitored based on the driving risk prediction result.

[0053] This embodiment proposes a driver status monitoring and intelligent intervention strategy in plateau areas that integrates multi-source data. The strategy integrates a variety of high-precision sensors and advanced computer vision technology to comprehensively collect complex parameters of the plateau environment and the driver's real-time driving status information. This solution can not only instantly evaluate and intelligently intervene in the driver's status to ensure driving safety, but also accurately predict the driving area of ​​unassisted vehicles in plateau environments by constructing refined mathematical models and simulation functions, thereby comprehensively improving the safety and efficiency of driving on plateau roads.

[0054] For the monitoring of plateau environment, this embodiment will focus on information such as environmental temperature and humidity, atmospheric pressure, oxygen concentration, wind speed and wind direction. For the driver's status monitoring, this embodiment will analyze from three aspects: physiological, psychological, and behavioral. The physiological aspect focuses on four indicators: driver's blood pressure, blood oxygen saturation, heart rate, and electrocardiogram signal; the psychological state can be analyzed from the driver's facial expression while driving, and the main judgment content is the driver's emotional changes; the behavioral aspect mainly analyzes whether the driver's operation is standardized, such as abnormal driving (sudden stop, speeding, frequent lane changes) and other behaviors.

[0055] The collection of plateau environmental information mainly relies on various sensors. The information collection for the plateau environment mainly consists of the following four parts:

[0056] (1) Data acquisition: Each of the four sensors, namely the temperature and humidity sensor, the oscillating cylinder air pressure sensor, the electrochemical oxygen sensor, and the wind speed and direction transmitter, has a sensitive element. When the plateau environment changes, the sensitive elements will undergo corresponding physical changes. These physical changes will be processed into electrical signals by the circuit or chip inside the sensor, thereby achieving the purpose of signal acquisition.

[0057] (2) Data preprocessing: This includes cleaning and converting the collected data.

[0058] (3) Information extraction: The digital signal in the sensor needs to be connected to the microprocessor through an interface for data processing to facilitate subsequent information transmission to the edge computing node.

[0059] (4) Plateau environment data analysis: Since the ultimate goal of this embodiment is to form a mathematical model of the driver's driving state changing with the plateau environment, different information sources should be combined in time to obtain a consistent explanation of the plateau environment changes, that is, multi-source data fusion. Considering the harsh plateau environment, a strong anti-interference ability is required in the data fusion process, so this embodiment adopts decision-level fusion.

[0060] Considering the changeable weather in the plateau, the data collected by the sensors are diverse, and there is no obvious pattern in their changes. Although the fuzzy logic algorithm has a certain flexibility in processing multi-source data and can handle various complex scenes, it has the limitation of low accuracy. Therefore, this embodiment adopts a combination of a neural network algorithm and a fuzzy logic algorithm. The neural network algorithm has a strong self-learning ability and can handle complex nonlinear relationships. Combined with the fuzzy logic algorithm, a fuzzy neural network can be formed to process uncertain and fuzzy information in the plateau environment and improve the robustness and adaptability of the system.

[0061] This embodiment selects a recurrent neural network (RNN) and fuzzy control for fusion. Each training sample of the recurrent neural network is a time series. There is a correlation between the input values ​​before and after the same training sample, and the sequence length of each sample may be different. During training, the input value of each moment in a sequence is first forward propagated, and then the gradient value of the parameter is calculated by back propagation and the parameter is updated. Combined with the correlation of temperature, air pressure, wind speed and direction changes in the plateau environment, it is most appropriate to select RNN to fuse environmental information.

[0062] The fusion of neural network and fuzzy logic algorithm adopts parallel mode, which takes RNN network as the dominant judgment basis. When the neural network cannot make accurate judgment, fuzzy logic control is used to assist the system in making judgment. At this point, the analysis of plateau environmental data M (M is a multidimensional space vector) is completed.

[0063] The following is an introduction to the collection and analysis of the driver's physiological-psychological-behavioral (also known as driving status) process, which is divided into four parts.

[0064] (1) Collection of physiological information: The air in the plateau environment is thin and low in oxygen, so drivers are very likely to experience breathing difficulties and hypoxia. In order to adapt to the environment, the driver's body will increase cardiac output to compensate for the hypoxia, which will lead to accelerated heart rate and increased blood pressure. Therefore, when the wearable device (a smart watch is selected in this embodiment) collects physiological information, the driver's blood pressure, blood oxygen saturation, heart rate and electrocardiogram signal are focused on.

[0065] ①The blood pressure measurement function is based on the pulse wave measurement principle of an optical sensor.

[0066] ②The blood oxygen saturation measurement principle used by the watch is also based on optical sensor technology. It uses an LED to emit light of a specific wavelength to illuminate the skin, and collects the reflected light through a photoelectric probe that receives light of a specific wavelength. When light shines into the blood, the light intensity changes. By detecting this difference, the blood oxygen saturation can be accurately measured.

[0067] ③The optical heart rate sensor on the back of the bracelet detects the expansion and contraction of human capillaries as the heart beats by sending light and receiving reflected light, and collects the fluctuations of the emitted light signal. By analyzing the frequency of the fluctuations of the light signal, the heart rate value can be calculated.

[0068] ④ The ECG (electrocardiogram) reading device on the wrist is based on the heart rate sensor. It forms a loop with the fingers and wrists through the electrode heart rate sensor on the crown and the optical heart rate sensor on the back to record the heart beat data. Among them, the electrode sensor can measure the electrical signal characteristics that control myocardial pumping.

[0069] (2) Collection of psychological information: Psychological state is mainly reflected in emotional expression. When the driver is in a negative mood, his attention is scattered and unfocused, which will prolong the driver's reaction time to emergencies and make it impossible to make correct decisions in time. If the driver's negative emotions can be detected and the system intervenes, it can help the driver avoid certain risks.

[0070] This embodiment uses a shallow convolutional neural network and a deep convolutional neural network in parallel (MV CNN) for facial expression analysis. MVCNN can solve the problems of CNN overfitting and gradient diffusion, and at the same time, establish the temporal information of the expression through the order of the facial image input model.

[0071] According to research, drivers' expressions are concentrated into six categories: happy, surprised, sad, fearful, angry, and calm. The six emotions are now divided into three categories:

[0072] ① Positive emotions: happiness, surprise;

[0073] ② Negative emotions: sadness, fear, anger;

[0074] ③ Neutral emotion: calm;

[0075] Among them, positive emotions are judged as suitable for driving and quantified as 1; negative emotions are judged as unfavorable for driving and quantified as -1; neutral emotions are judged as normal driving and quantified as 0. Since human emotions are changeable, and the same emotions can be strong or weak, the change from negative emotions to positive emotions is quantified as a continuous change, that is, an uninterrupted change from -1 to 1. Among them, the emotional state between 0 and 1 is judged as suitable for driving, and between -1 and 0 is judged as unsuitable for driving.

[0076] (3) Collection of behavioral information: Due to the complex terrain in plateau areas, there are a large number of sharp turns, steep slopes, continuous ups and downs, etc. on the road, and the driving speed is usually difficult to control. Speeding will cause the driver to have insufficient time to react to emergencies, and emergency stops will cause the vehicle's engine to burn incompletely, resulting in failures such as flameout and lack of power. Secondly, the roads in plateau areas are usually narrow and complex, and frequent lane changes can easily cause vehicle rollovers, collisions, and other accidents. Therefore, this embodiment uses a Hall-type wheel speed sensor to detect vehicle speeding and emergency stops, and uses a laser radar with wireless transmission function to detect vehicle lane changes.

[0077] The Hall sensor data acquisition module collects the analog signal output by the Hall sensor through the ADC (analog-to-digital converter) inside the microcontroller, and processes the signal through a digital filtering algorithm to improve the data quality. The data processing module calculates the wheel speed based on the collected pulse signal and then obtains the vehicle speed value. By measuring the speed, it can be known whether the vehicle is speeding or making an emergency stop.

[0078] Since the position measurement of the car changing lanes requires a certain degree of accuracy and is not affected by the plateau environment, this embodiment uses laser radar for positioning. Laser radar is a radar system that emits a laser beam to detect the position, speed and other characteristic quantities of the target. Its working principle is to emit a detection signal (laser beam) to the target, and then compare the received signal reflected from the target (target echo) with the transmitted signal. After appropriate processing, the relevant information of the target can be obtained.

[0079] This embodiment uses the time-of-flight method to measure distance. The time-of-flight method uses the speed of light to measure distance. If the laser transmitter is at t 0 A laser beam is emitted at a certain moment, hitting the object and reflecting. 1 When the laser receiver receives the time, the measured distance D can be calculated using the following expression:

[0080]

[0081] Where C represents 3×10 8 m / s. Since the speed of light is too fast, the timer needs to be highly accurate. Therefore, in practical applications, phase measurement is usually used instead of direct time measurement. Time can be calculated using phase difference:

[0082]

[0083] Where Δt = t 1 -t 0 , is the phase difference of the laser transmitting and receiving waveforms, and the frequency of one cycle is f m .

[0084] (4) Driver driving status data analysis:

[0085] After the above three steps, various data of the driver's driving status have been collected. The collected data are also fused by combining fuzzy logic control with neural network to obtain comprehensive driving status data.

[0086] In the fourth part of plateau environmental information collection: "Plateau environmental data analysis", this embodiment selects a combination of fuzzy logic control and recurrent neural network to analyze plateau environmental data due to the particularity of the mutual influence between various environmental factors.

[0087] In the process of collecting the driver's driving status, the collected information is of many types and large in quantity, and a neural network with better performance and more stable processing of complex problems is needed to combine with fuzzy logic control for data analysis. Therefore, when analyzing and processing driving status information, this embodiment selects the fusion of BP neural network and fuzzy logic control, and the fusion process is as follows:

[0088] ① After determining the number of layers and nodes of the input layer, hidden layer, and output layer, the input layer of the neural network transmits the data information of each device, and then passes through the input layer, hidden layer, and output layer in turn to achieve forward propagation of the data information;

[0089] ②For the first forward propagation process, the weights of each layer of the neural network remain unchanged, and the neurons in the current propagation layer only act on the neurons in the next layer; if the result of the first forward propagation does not meet the requirements, an error value will be generated, and the neural network will enable error back propagation at this time;

[0090] ③ When the error obtained by the first forward propagation is not within the expected error range, the error will be corrected from back to front. The process of neural network error back propagation is the opposite of the process of neural network forward propagation;

[0091] ④ Repeatedly perform forward propagation and error back propagation until the error value of the output value is minimized and then stop propagation;

[0092] ⑤ When the results of the BP neural network and the fuzzy logic control decision-making are consistent, the system can determine that the final result is the result of the two; when the results of the BP neural network and the fuzzy logic control decision-making are inconsistent, the decision result of the BP neural network is given priority, and the decision made by the fuzzy logic control is used as an auxiliary judgment. At this point, the analysis of the driver's driving state data N (N is a multidimensional space vector) is completed.

[0093] The intervention in the driving process of this embodiment is mainly reflected in the early warning of physiological state and the intervention of negative emotions and improper driving behavior. Among them, the early warning of physiological state mainly uses the Bayesian prediction algorithm. In Bayesian prediction, there is usually a prior probability distribution, which represents our initial estimate of the occurrence of the event (this embodiment sets the prior probability distribution according to the initial physiological state of the driver). After updating the prior probability distribution according to the collected data, a posterior probability distribution can be obtained. The posterior probability distribution represents an estimate of the probability of an event occurring after the data is observed. When the system detects that the posterior probability reaches 95% (that is, 95% of the driver's physiological tolerance), it will remind the driver to pay attention to rest, ensure the safety of the driving process, and avoid accidents.

[0094] When negative emotions are detected in the driver, intervention mainly uses methods such as playing music and adjusting the color of the lights in the car to help soothe the driver's emotions. Studies have shown that gray and white tones account for the majority of psychological changes (i.e. emotional changes) affecting drivers. Therefore, vehicle lights should avoid this color and try to choose cool colors such as blue and green to help increase the excitement of the driver's emotions. In terms of music, songs with a strong sense of rhythm and obvious drum beats can be played to regulate the driver's sadness, and soothing and soft melodies can be played to regulate the driver's fear and anger.

[0095] exist Figure 1The specific development process of this embodiment can be seen in: Four sensors, namely temperature and humidity sensor, vibrating cylinder air pressure sensor, electrochemical oxygen sensor, and wind speed and direction transmitter, are used to collect plateau environmental information; wearable devices (smart watches), computer vision systems, lidar, and Hall sensors are used to collect the driver's driving status. The information collected by the two can be used to obtain two sets of multidimensional vectors (M, N) that can represent changes in the plateau environment and driving status through edge computing. The system intervention is mainly aimed at the driver's driving status. For example, the method of early prediction is adopted for the physiological state to remind the driver to pay attention to rest and adapt to the environment; the method of changing the light color and playing music is adopted for the psychological (emotional) state to help the driver regulate his emotions; and voice prompts are adopted for abnormal driving to inform the driver of the dangerous consequences of abnormal driving. At the same time, for the two sets of multidimensional vectors, this embodiment adopts the method of training mathematical models to establish a functional change relationship between the two, which can be used as a reference for subsequent drivers without equipment who have plateau driving needs.

[0096] Figure 1 The edge computing involved has the following steps:

[0097] ① Terminal nodes: composed of various IoT devices ( Figure 1 It is composed of all sensors and wearable devices in the network, and its main function is to collect raw data and report it.

[0098] ②Edge computing nodes: Edge computing nodes reasonably deploy and allocate the computing and storage capabilities of network edge nodes. Figure 1 It is reflected in the microcontroller, computer vision technology, etc. to realize basic service response.

[0099] ③ Network nodes: responsible for uploading useful data processed by edge computing nodes to cloud computing nodes for analysis and processing. Figure 1 It is reflected in wireless transmission.

[0100] ④ Cloud computing nodes: The reported data of the edge computing layer will be permanently stored in the cloud computing nodes. At the same time, the analysis tasks that the edge computing nodes cannot handle and the processing tasks of comprehensive global information still need to be completed in the cloud computing nodes. In addition, the cloud computing nodes can also dynamically adjust the deployment strategy and algorithm of the edge computing layer according to the distribution of network resources. This patent adopts a combination of neural network algorithm and fuzzy logic algorithm.

[0101] It should be noted that in plateau environments, extreme weather will pose some challenges to edge computing (especially cloud computing nodes). Therefore, this embodiment uses special electronic devices or batteries (lithium titanate batteries) to ensure the stability of the work, and uses a distributed computing architecture to improve the reliability and fault tolerance of the system.

[0102] Figure 2 , Figure 3This is the installation location of the related equipment used in this embodiment.

[0103] like Figure 4 As shown in the figure, when collecting the temperature and humidity of the plateau environment, the temperature and humidity values ​​of the environment can be read in real time by connecting the sensor to the microcontroller. Similarly, the vibrating cylinder air pressure sensor, electrochemical oxygen sensor, and wind speed and direction transmitter also read the real-time values ​​through the above steps. The network node (wireless transmission) sends this data to the cloud computing node for data fusion.

[0104] The data fusion process is as follows:

[0105] ① Modal data preprocessing: converting data of different modalities into the same format;

[0106] ② Modal data feature extraction: In addition to receiving data from the previous layer, each hidden layer also receives data from the previous time point of the current layer;

[0107] ③ Modal feature fusion: merge data features of different modalities into the same feature space.

[0108] In order to solve the problem of gradient disappearance that often occurs in recurrent neural networks, this embodiment chooses to use LSTM (Long Short-Term Memory) neuron nodes in the hidden layer.

[0109] exist Figure 5 In the physiological detection part of the driver, this embodiment uses a wearable device that can upload data to a cloud server for subsequent data analysis and processing. The parameters that can be monitored include blood pressure, blood oxygen saturation, heart rate and electrocardiogram signals. The wearable device used realizes the functions of collecting, processing, displaying, storing and transmitting these parameters.

[0110] Figure 5 The specific steps of the psychological testing part are:

[0111] ①Facial image acquisition;

[0112] ② Use MVCNN (shallow and deep neural network parallel structure) for expression recognition. The network structure is as follows Figure 6 As shown in the figure: In this framework, the shallow convolutional neural network is mainly used to extract the significant features (obvious features) of facial expressions, while providing basic optimization parameters of the model to speed up the convergence of the entire model. The deep convolutional neural network is mainly used to extract the deep features of facial expressions (features that are difficult to identify with the naked eye), enrich the feature information of the image in the model, and improve the recognition accuracy of the model.

[0113] The main function of the fully connected layer is to stretch the image into a one-dimensional vector and then classify it. In classification tasks, the fully connected layer is usually connected to one or more classifiers, such as the Softmax classifier. The classifier can predict the category of the image based on the input feature vector. The outputs of the two neural networks are finally merged in the fully connected layer FC, and the Softmax loss function is used as the input of the subsequent fully connected layer Softmax, and the output of this layer is the classification of six expressions: happy, surprised, sad, fearful, angry, and calm.

[0114] The continuous changes of expressions on the time axis can reflect the changes of current emotions to a certain extent. In this model, the temporal information is expressed as follows:

[0115] img k →MVCNN model→[softmax(img) i t k ]t k =k

[0116] In the formula, img k It represents the kth facial image input to the CNN (convolutional neural network) expression recognition model. MVCNNmodel is the expression recognition model. [softmax(img) i t k ] represents the recognition result matrix with time series information. Among them, softmax(img) i Refers to img after being processed by the Softmax function k The probability of being identified as class i, t k It means img k The time series information that is identified as category i has the same value as k. In relation to the actual classification of expression recognition, img k After the MVCNN (shallow and deep neural network parallel structure) expression recognition model recognizes, the final recognition result with time sequence information can be expressed as follows:

[0117]

[0118] This formula is the final output of the established MVCNN (shallow and deep neural network parallel structure) expression recognition model.

[0119] Among them, softmax(img k ) 0 …softmax(img k ) 5They represent the recognition probability of different expressions, corresponding to 0-Anger, 1-Calm, 2-Fear, 3-Happy, 4-Sad, and 5-Surprised.

[0120] ③Emotional index of facial expression:

[0121] By judging softmax(img k ) 0 and softmax(img k ) 6 The recognition probability of the two facial expressions with the highest probability in the middle facial expression: softmax(img k ) x and softmax(img k ) y , then img k The degree of expression and emotion can be expressed as:

[0122]

[0123] In the formula, They represent expression images img respectively k The emotional index size and expression image img k Emotion index size of facial expressions. and The emotion indexes corresponding to the conversion of the recognized expression at the last moment to facial expression x and the conversion of the recognized expression at the last moment to facial expression y are obtained by querying the facial expression conversion table. softmax(img k ) x and They are respectively expression images img k The recognition probability of facial expression x,y.

[0124] For the obtained expression image img k Sentiment Index The continuous emotion index is obtained by continuously integrating the time period during which the expression occurs. In addition, a filtering link needs to be introduced to solve the problem related to time and expression. By attenuating the emotional index in different time periods, the driver's emotional index can be controlled within a reasonable range. The formula is as follows:

[0125]

[0126] In the formula, ε k It means t k The driver's emotional index at the moment (i.e., the current moment), that is, the emotion recognition algorithm recognizes the expression image imgk The resulting emotional state of the driver. k-1 It means t k-1 The emotional index of the driver at the moment. α is the first-order filter coefficient, and its size determines the decay speed of the emotional index of the expressions sorted by time. The larger the α value, the slower the decay speed of the emotional index of the recognized expressions sorted by time, and the emotional index of the currently recognized expression accounts for a smaller proportion of the driver's emotional index.

[0127] exist Figure 5 In the driving behavior detection part, this embodiment selects Hall wheel speed sensor and laser radar. The working principle of Hall sensor is based on the Hall effect. When current passes through a conductor in a magnetic field, a potential difference is generated at both ends of the conductor. The Hall sensor uses this principle to measure the change of the magnetic field and convert it into an electrical signal output. The subsequent process is similar to Figure 4 The sensor process is similar to that of the sensor. The electrical signal is converted into a digital signal through a digital-to-analog converter, and finally the data is transmitted to the cloud computing node through a microcontroller.

[0128] Frequent lane changes are mainly based on the data set of spatial points obtained by the laser radar through scanning the environment, that is, the point cloud. When the position of the vehicle is found to be frequently moving left or right, it is judged that the vehicle is changing lanes frequently.

[0129] When using point cloud data for positioning, we must first extract the features of the target objects in the surrounding scene, and use these features and the relative distance information obtained to build a small map to know the relative initial position of the vehicle. Then, point cloud map matching is performed based on these features to obtain the relative positions between each point cloud. After the point cloud data is stored, it can be uploaded to the cloud computing node through the API (application programming interface) for data analysis and visualization for subsequent lane change judgment.

[0130] Figure 4 , Figure 5 The cloud computing node part has been explained in the previous article and will not be repeated here.

[0131] The intervention of this embodiment is divided into three parts: intervention on physiological state ( Figure 7 ), intervention on psychological state ( Figure 8 ) and intervention in driving behavior ( Fig. 9 ).

[0132] exist Figure 7In the process, the prior probability is determined by analyzing the driver's daily driving status to obtain the possibility of reaching the physiological state threshold; then the conditional probability is calculated based on the continuously updated physiological data of driving in the plateau environment, that is, the probability that the driver reaches the physiological threshold when driving in the plateau environment; the prior probability and conditional probability are combined using Bayes' theorem to calculate the posterior probability. The posterior probability can predict the upcoming situation so that the driver can adjust the state to adapt to the environment in advance.

[0133] exist Figure 8 In the process of adjusting the driver's emotions, combined with the quantitative value of emotions, when the quantitative value is in the unsuitable driving range of -1 to 0, the driver's emotions can be adjusted by playing music and adjusting the color of the lights in the car. For example, blue and green lights combined with music with strong rhythm can be used to adjust the emotions of fear and anger; red lights and music with strong or high pitch can be used to adjust the emotions of sadness.

[0134] exist Fig. 9 In the current system, the intervention for abnormal driving mainly adopts the method of voice reminder, which can improve the driver's safety awareness by reminding the driver of the dangerous consequences of abnormal driving.

[0135] In addition to detecting the safety status of drivers of vehicles equipped with equipment driving on the plateau, this system can also collect information about the changes in the driver's driving status as the plateau environment changes, forming a big data model, which can serve as a reference for drivers who do not have the conditions to equip their vehicles with the system in the future. The big data model can calculate the driver's acceptable plateau environmental limit based on his physiological condition, helping the driver avoid some potential risks.

[0136] After multi-source fusion of plateau environment data and driver driving status data, two sets of vectors M and N are obtained respectively:

[0137] M=[x 1 ,x 2 ,x 3 …],N=[y 1 ,y 2 ,y 3 …]

[0138] These two sets of vectors have a functional relationship of N = f(M). Now let's fit them:

[0139] The methods of fitting data include linear regression, polynomial regression and logistic regression. Among them, linear regression is suitable for fitting linear data relationships; polynomial regression is suitable for fitting nonlinear data relationships. This method describes the relationship between two variables by fitting polynomials to independent variables; the assumption of the logistic regression model is that there is a nonlinear relationship between the dependent variable and the independent variable, and the error term conforms to the binomial distribution. After comprehensive analysis, this embodiment selects polynomial regression to fit the two sets of data of the driver's driving status and the plateau environment changes.

[0140] The main idea of ​​polynomial regression is to fit the polynomial regression equation through historical data, and use the polynomial regression equation to predict new data. The equation combined with the previous vector can be expressed as:

[0141]

[0142] In the formula, M is the plateau environment fusion data, N is the driving state fusion data, and a n is the weight, x i (i=1,2,3…n) represents the multidimensional vector after the fusion of plateau environmental data, that is, M=(x 1 , x 2 , x 3 …x n ), y i (i=1,2,3…n) represents the multidimensional vector after the driver’s driving status data is fused, that is, N=(y 1 ,y 2 ,y 3 …y n ), N = f(M) represents the functional relationship between plateau environmental changes and driving conditions. i With x i Correspondingly, each calculation results in a y i The values ​​of (i = 1, 2, 3...) are placed in the vector N. Once all calculations are completed, the physiological state data that may occur when driving in a plateau environment can be predicted. To obtain a polynomial regression model that perfectly fits the target data set, the essence is to solve the weight a of each feature independent variable n (n=1, 2, 3...). The specific fitting process is as follows Fig.10 As shown:

[0143] ①First, input the M and N vectors into the curve model:

[0144]

[0145] ② Calculate the sum of squares of deviations from each point to the fitting curve:

[0146]

[0147] ③ For the right side of the squared deviation equation, n Find the partial derivative:

[0148]

[0149] ④Simplify the above equation to get the matrix:

[0150]

[0151] Extract the matrix:

[0152]

[0153] ⑤ Matrix calculation to obtain the actual weight a n :

[0154] X*A n =Y

[0155] A n =X -1 *Y

[0156] Calculate the weight a n After that, the functional relationship is determined. According to the functional relationship, drivers who are driving on the plateau for the first time without a real-time detection system can input their basic physiological information and usual driving habits, and then calculate the range of the plateau area suitable for driving through functional calculations. When the system collects enough data, the more accurate the prediction of the functional model is, the higher the reference value is.

[0157] The model provided in this embodiment has the following beneficial effects and significance:

[0158] The system can enhance the safety of driving in plateau environments. By integrating high-precision sensors and advanced visual technology, it monitors plateau environmental parameters and driver status in real time. Once any data that deviates from safety standards is detected, it immediately issues a warning to the driver, effectively preventing potential high-risk situations and ensuring safe and worry-free driving in plateaus. At the same time, it provides road condition predictions and driving suggestions to improve overall driving safety and efficiency.

[0159] By extensively collecting detailed data on driving status and plateau environmental changes, this embodiment constructs a real and reliable database. Based on this database, an accurate plateau environment-driving status mathematical model can be trained. For drivers in vehicles that are not equipped with real-time monitoring equipment, they can input their personal physiological parameters into the model before driving on the plateau to obtain a predicted plateau environmental tolerance limit. This predicted value will serve as an extremely valuable reference, giving drivers stronger predictive ability, so that they can make more appropriate decisions during plateau driving and significantly improve driving safety in plateau areas.

[0160] By studying how the driver's driving state changes with the environment in plateau areas, it can provide useful reference for automobile design. For example, automobile manufacturers can optimize the performance and oxygen supply system of the car based on these research results to adapt to the special environment of the plateau and improve the driver's driving experience.

[0161] In general, in-depth research on the dynamic model of drivers' driving status changing with the environment in plateau areas is of immeasurable value in significantly improving driving safety and traffic efficiency. This not only deepens drivers' understanding of the particularity of plateau traffic, but also provides a solid theoretical basis for formulating targeted plateau driving strategies. It also becomes a scientific support for the formulation of traffic safety education and prevention strategies, and effectively promotes the overall progress of traffic safety in plateau areas.

[0162] The above is only a preferred specific implementation of the present application, but the protection scope of the present application is not limited thereto. Any changes or substitutions that can be easily thought of by a person skilled in the art within the technical scope disclosed in the present application should be included in the protection scope of the present application. Therefore, the protection scope of the present application should be based on the protection scope of the claims.

Claims

1. A driver status monitoring and intervention method in a plateau environment, characterized in that: include: Acquire plateau environment data and driver's driving status data; The plateau environment data and the driving state data are transmitted to a cloud computing node for multi-source data fusion processing to obtain corresponding plateau environment fusion data and driving state fusion data, and a functional relationship between plateau environment changes and driving states is constructed based on the plateau environment fusion data and the driving state fusion data; Based on the constructed functional relationship, real-time risk prediction of the driver to be monitored is performed to obtain a driving risk prediction result, and intervention is performed on the driving state of the driver to be monitored based on the driving risk prediction result.

2. The driver status monitoring and intervention method in a plateau environment according to claim 1 is characterized in that: The plateau environmental data include temperature and humidity data, air pressure data, oxygen content data and wind speed and direction data.

3. The driver status monitoring and intervention method in a plateau environment according to claim 1 is characterized in that: The driving status data includes the driver's physiological information data, psychological information data and driving behavior data.

4. The driver status monitoring and intervention method in a plateau environment according to claim 3 is characterized in that: The process of obtaining the psychological information data specifically includes: Acquiring facial image data of the driver; The facial image data is input into a pre-trained expression recognition model to perform facial expression analysis to obtain emotional state data, and the emotional state data is uploaded to a cloud computing node as psychological information data; wherein the expression recognition model is constructed based on an MVCNN network.

5. The driver status monitoring and intervention method in a plateau environment according to claim 3 is characterized in that: The process of acquiring the driving behavior data specifically includes: The speeding and emergency stop conditions of the vehicle are detected based on the Hall-type wheel speed sensor, and the lane change conditions of the vehicle are detected using the laser radar. The detected speeding, emergency stop and lane change conditions are uploaded to the cloud computing node as driving behavior data.

6. The driver status monitoring and intervention method in a plateau environment according to claim 1 is characterized in that: Before transmitting the plateau environment data and the driving status data to the cloud computing node for multi-source data fusion processing, the plateau environment data is also cleaned and format converted to obtain pre-processed plateau environment data, and the pre-processed plateau environment data is transmitted to the cloud computing node through a microprocessor for multi-source data fusion processing.

7. The driver status monitoring and intervention method in a plateau environment according to claim 1 is characterized in that: The transmitting the plateau environment data and the driving status data to a cloud computing node for multi-source data fusion processing specifically includes: The plateau environment data and the driving status data are respectively input into a multi-source data fusion module for data fusion to obtain corresponding plateau environment fusion data and driving status fusion data; wherein the multi-source data fusion module includes a BP neural network prediction module and a fuzzy logic control decision module set in parallel.

8. The driver status monitoring and intervention method in a plateau environment according to claim 7 is characterized in that: The processing process of the multi-source data fusion module specifically includes: The data to be input is input into the BP neural network prediction module and the fuzzy logic control decision module for data fusion. When the output results of the BP neural network prediction module and the fuzzy logic control decision module are consistent, the optimal fusion data is output; when the output results of the BP neural network prediction module and the fuzzy logic control decision module are inconsistent, the output result of the BP neural network prediction module is used as the main decision, and the output result of the fuzzy logic control decision module is used as the auxiliary decision, and the optimal fusion data is determined based on the main decision and the auxiliary decision.

9. The driver status monitoring and intervention method in a plateau environment according to claim 1 is characterized in that: The constructing of the functional relationship between the plateau environment change and the driving state specifically includes: Polynomial fitting is performed on the plateau environment fusion data and the driving state fusion data to obtain a functional relationship between the plateau environment change and the driving state. The specific calculation formula is: In the formula, M is the plateau environment fusion data, N is the driving state fusion data, and a n is the weight, x i (i=1,2,3…n) represents the multidimensional vector after the fusion of plateau environmental data, that is, M=(x1,x2,x3…x n ), y i (i=1,2,3…n) represents the multidimensional vector after the driver’s driving status data is fused, that is, N=(y1,y2,y3…y n ), N = f(M) represents the functional relationship between plateau environment changes and driving conditions.

10. The driver status monitoring and intervention method in a plateau environment according to claim 1, characterized in that: Intervening in the driving state of the driver to be monitored based on the driving risk prediction result specifically includes: Early warning intervention of the driver's physiological state to be monitored: Based on Bayesian theorem, the driver's initial physiological state is taken as a priori probability, and the physiological state of the driver to be monitored when driving in a plateau environment is taken as a conditional probability. The posterior probability is calculated based on the priori probability and the conditional probability. When the posterior probability is greater than a preset threshold, an early warning prompt is given to the driver. Intervene the negative emotions of the monitored driver: When the monitored driver shows negative emotions, the driver's emotions can be alleviated by playing music and adjusting the color of the lights in the car.

Images