Passenger carsickness degree evaluation method and system
Through the multimodal feature fusion method, combining facial expressions, physiological data and vehicle motion parameters, the degree of passenger motion sickness is accurately identified, which solves the problem of insufficient single mode recognition in the existing technology, and achieves high accuracy and intelligence of passenger motion sickness assessment. Slow down control.
Patent Information
- Application Number
- CN202411038052.8
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-07-31
- Publication Date
- 2024-12-03
AI Technical Summary
The prior art is difficult to accurately identify the degree of motion sickness of passengers. Single modal features such as posture, physiological data or facial expressions are not sufficient to effectively evaluate motion sickness status, and physiological data have poor anti-motion interference.
By using multimodal feature fusion method, the facial expression and physiological data characteristics are extracted by collecting subjective and objective data of occupants in different road scenarios, and by using residual neural networks, long and short-term memory networks and one-dimensional convolutional layers, they are combined with vehicles. The motion parameter characteristics are weighted and feature spliced, and the recognition model is input to output motion sickness levels.
Accurate assessment of the degree of motion sickness of passengers is achieved, the accuracy and reliability of the assessment is improved, and the vehicle driving dynamics parameters are intelligently adjusted according to the evaluation results to slow down the symptoms of motion sickness.
Smart Images

Figure CN119066545A_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to the field of motion sickness analysis, and in particular to a method and system for evaluating the degree of motion sickness of passengers. Background Art
[0002] The growth of new energy vehicle ownership cannot be ignored. As people's demand for "intelligence, interconnection, and lightweight" becomes higher and higher, more and more people will choose electric vehicles when buying cars or traveling. While electric vehicles provide people with various intelligent and convenient experiences, people's requirements for the comfort of traveling are also getting higher and higher. Many passengers reported that they had adverse symptoms such as sweating, dizziness, pale face, nausea and even vomiting during the driving of the vehicle. This kind of motion sickness is called motion sickness. In order to improve the phenomenon of motion sickness and improve the comfort of passengers, it is necessary to identify whether the passengers are motion sick and the level of motion sickness.
[0003] At present, in order to identify motion sickness, some methods only use rich posture features to identify motion sickness, but ignore the interference caused by passengers' fatigue or simply uncomfortable sitting posture, so body posture is not suitable as a single modality to identify motion sickness. Some methods identify motion sickness by monitoring indicators such as passengers' heart rate, blood pressure or skin electrodermal activity, but physiological data has weak resistance to motion interference. Passengers drinking water, talking or experiencing some large fluctuations will have a significant impact on the data. Others use facial expressions to observe a series of distinct changes in the face of a person with motion sickness. However, considering that facial expressions reflect not only the state of motion sickness, but also the rich and diverse emotions and moods of humans, identifying motion sickness from facial expressions alone is not a suitable method.
[0004] Therefore, in order to solve the above problems, a method and system for assessing the degree of motion sickness of passengers is needed, which can comprehensively consider the various manifestations of passengers' motion sickness, fuse the features of different modalities, and accurately identify the degree of motion sickness of passengers. Summary of the invention
[0005] In view of this, the purpose of the present invention is to overcome the defects in the prior art and provide a method and system for assessing the degree of motion sickness of passengers, which can comprehensively consider various manifestations of passengers' motion sickness, fuse features of different modes, and accurately identify the degree of motion sickness of passengers.
[0006] The method for assessing the degree of motion sickness of passengers of the present invention comprises the following steps:
[0007] S1. Collect subjective data and objective data of passengers in different road scenarios; use subjective data as labels to annotate objective data;
[0008] S2. Extract features from the labeled objective data and extract feature information of vehicle motion parameters;
[0009] S3. Fusing the feature information extracted in step S2 to obtain fused feature information;
[0010] S4. Input the fused feature information into the recognition model and output the recognition result of the occupant's motion sickness level.
[0011] Furthermore, the subjective data includes motion sickness level MISC score; the objective data includes physiological data and facial expression data.
[0012] Furthermore, feature extraction is performed on the labeled objective data, specifically including:
[0013] The facial expression data is subjected to feature extraction through residual neural network and long short-term memory network to obtain a feature vector with a sequence length of m;
[0014] After the physiological data passes through n one-dimensional convolutional layers and an adaptive average pooling layer of size k, it is processed into a feature sequence with s channels and a length of k. Then, the output dimension is changed from [N, s, 1] to [N, s] through a dimensionality reduction operation, and the dimension of length 1 is removed to obtain an s-dimensional feature vector.
[0015] Further, the vehicle motion parameter characteristic information is extracted, including:
[0016] After passing through n one-dimensional convolutional layers and an adaptive average pooling layer of size k, the vehicle motion parameters are processed into a feature sequence with s / 2 channels and a length of k. Then, the output dimension is changed from [N, s / 2, 1] to [N, s / 2] through a dimensionality reduction operation, and the dimension of length 1 is removed to obtain an s / 2-dimensional feature vector.
[0017] Further, the feature information extracted in step S2 is fused, specifically including:
[0018] Weights are configured for physiological data, facial expression data, and vehicle motion parameters respectively, and weighted processing is performed. Then, the feature vectors from different data sources are concatenated along the feature dimension to form a feature vector.
[0019] Furthermore, the characteristic vector corresponding to the physiological data includes physiological characteristic indicators; the physiological characteristic indicators include the mean of electrodermal activation, the maximum of electrodermal activation, the minimum of electrodermal activation, the median of electrodermal activation, the standard deviation of electrodermal activation, the range of electrodermal activation, the skewness of electrodermal activation, the average heart rate of electrocardiogram, the mean of adjacent RR intervals, the standard deviation of adjacent RR intervals, the root mean square difference of adjacent RR intervals, the high-frequency heart rate variability, the percentage of adjacent RR intervals greater than 50ms, the mean of respiratory rate, the mean of respiratory variability, the coefficient of respiratory variability and the standard deviation of consecutive RR intervals.
[0020] Furthermore, the method further includes: performing motion sickness relief processing according to the motion sickness level of the occupant:
[0021] When the occupant is not in a motion sickness state, the vehicle driving dynamics parameters are adjusted so that the vehicle driving dynamics parameters are below the mild motion sickness index value;
[0022] When the occupant is in a state of mild motion sickness, the vehicle driving dynamics parameters are adjusted so that the vehicle driving dynamics parameters are below the non-motion sickness index value;
[0023] When the occupants are in a state of severe motion sickness, the vehicle's driving dynamics parameters are no longer adjusted and the vehicle is stopped for rest.
[0024] Furthermore, in a straight road scenario, the vehicle driving dynamics parameters include a maximum longitudinal acceleration, a longitudinal acceleration change value, and a maximum longitudinal driving speed;
[0025] In a curved road condition scenario, the vehicle driving dynamics parameters include a longitudinal acceleration change value and a vertical angular velocity average value.
[0026] A system for assessing the degree of motion sickness of passengers, comprising a collection unit, a feature extraction unit, a feature fusion unit and a motion sickness recognition unit;
[0027] The collection unit is used to collect subjective data and objective data of passengers in different road scenes; wherein the objective data is labeled with the subjective data;
[0028] The feature extraction unit is used to extract features from the annotated objective data and extract vehicle motion parameter feature information;
[0029] The feature fusion unit is used to fuse the extracted feature information to obtain fused feature information;
[0030] The motion sickness recognition unit is used to input the fused feature information into the recognition model and output the motion sickness level recognition result of the occupant.
[0031] Furthermore, it also includes a motion sickness relief unit; the motion sickness relief unit is used to perform motion sickness relief processing according to the motion sickness level of the occupant.
[0032] The beneficial effects of the present invention are as follows: a method and system for assessing the degree of motion sickness of an occupant disclosed in the present invention collects physiological indicators, facial expressions, etc. of the occupant through sensor technology, cameras and other equipment, and then conducts a comprehensive assessment through data analysis and machine learning algorithms. A motion sickness assessment model is established based on the collected data, the influence of different factors on motion sickness is considered, and the degree of motion sickness of the occupant is accurately assessed based on the results output by the model. The model can be continuously optimized and updated, thereby improving the accuracy and reliability of the assessment. BRIEF DESCRIPTION OF THE DRAWINGS
[0033] The present invention will be further described below in conjunction with the accompanying drawings and embodiments:
[0034] Figure 1 It is a schematic flow chart of the motion sickness degree assessment method of the present invention;
[0035] Figure 2 A schematic diagram of a road scene of the present invention;
[0036] Figure 3 It is a motion sickness experiment operation flow chart of the present invention;
[0037] Figure 4 It is a schematic diagram of the data collection process of the present invention;
[0038] Figure 5 It is a schematic diagram of motion sickness susceptibility weight output of the present invention;
[0039] Figure 6 It is a schematic diagram of the feature fusion process of the present invention;
[0040] Figure 7 It is a schematic diagram of motion sickness level identification of the present invention. DETAILED DESCRIPTION
[0041] The present invention is further described below in conjunction with the accompanying drawings, as shown in the drawings:
[0042] This embodiment discloses a method for assessing the degree of motion sickness of an occupant, comprising the following steps:
[0043] S1. Collect subjective data and objective data of passengers in different road scenarios; use subjective data as labels to annotate objective data;
[0044] S2. Extract features from the labeled objective data and extract feature information of vehicle motion parameters;
[0045] S3. Fusing the feature information extracted in step S2 to obtain fused feature information;
[0046] S4. Input the fused feature information into the recognition model and output the recognition result of the occupant's motion sickness level.
[0047] In this embodiment, considering the actual use scenarios of the vehicle, typical road scenarios that are likely to cause motion sickness to passengers are selected, such as one-way turns, S-shaped curves, uphill and downhill, and acceleration and deceleration. In order to ensure the safety of the experiment, road conditions with less traffic volume are selected. After comprehensive consideration, road scenarios such as Figure 2The overall experimental route is 17 kilometers, and the estimated driving time is about 35 minutes, which is in line with the daily commuting time of electric vehicles. It also contains common non-congested and congested sections in life, and winding mountain roads that are prone to motion sickness, meeting the needs of experimental motion sickness induction. The road can be changed as long as it meets the requirements of successfully inducing motion sickness symptoms in passengers.
[0048] Selection of subjects (passengers): Considering factors such as gender, age, and motion sickness sensitivity, 40 subjects who are prone to motion sickness symptoms were selected. All subjects were adults and volunteered to participate in the experiment. Experimental process: The purpose of the experiment is to collect the motion sickness levels of subjects with different motion sickness characteristics after driving with the same vehicle dynamic parameters. The specific experimental process is as follows: Figure 3 The experiment is divided into three stages: pre-experimental preparation, formal experiment, and post-experimental questionnaire interview.
[0049] Pre-experimental preparation stage: ①Test process description: Guide the subjects to enter, explain the purpose and content of the experiment in detail, and introduce the overall process of the experiment. Record the basic personal information of the subjects through the questionnaire, including name, age, gender, etc.; ② The subjects are familiar with the questionnaire, sign the informed consent form, explain the meaning of the MISC scale to the subjects, and make them familiar with the descriptions corresponding to each degree of motion sickness, so that the subjects can quickly respond according to their own motion sickness status during the experiment and assess the degree of motion sickness. And read and sign the informed consent form carefully; ③ Wear the corresponding equipment: The main experimenter wears physiological instruments and other equipment for the subjects.
[0050] Formal experiment: ④Static data recording: After the subject wears the device, he / she enters the back seat and adjusts to a comfortable sitting position, keeping his / her emotional state calm. The main experimenter 2 collects 1 minute of device data as the benchmark for all data, which is ready for subsequent use. ⑤Dynamic driving process: After the static data collection is completed, the driver starts the vehicle and drives along the specified route. During the driving process, the subject sits in the back seat of the car as required, and the main experimenter 2 records the objective data of the whole process. ⑥Questionnaire recording and objective data marking at the task point: When the driver drives through the task point, the main experimenter 1 consults the subject's current MISC level twice and records it (before and after the task point), and the main experimenter 2 marks it in the physiological instrument software (1-before the task point; 2-after the task point). ⑦The end of the experiment, record the data, and remove the device: The driver drives the vehicle to the end point, the main experimenter 1 consults the subject's MISC level, and the main experimenter 2 collects static data for 1 minute after the experiment, then ends the experiment and removes the device from the subject. Among them, if the motion sickness level of the MISC scale reaches 8 or the subject is physically unwell, the experiment will be terminated immediately.
[0051] Post-experimental questionnaire interview:⑧After the experiment, the subjects filled in the MSQ symptom scale questionnaire and were interviewed about their feelings during the entire experiment. The MSQ is designed to assess and record various physical symptoms and discomforts of individuals in order to better understand the degree of motion sickness of the subjects.
[0052] The subjective data includes the MISC score of motion sickness level; the objective data includes physiological data and facial expression data. The whole experimental road scene is divided into 6 typical road scene tasks. Before and after each road condition, the subjects answer the subjective motion sickness level MISC score according to the motion sickness symptoms at that time. The objective motion sickness physiological data and facial expression data are collected throughout the process and the corresponding marking points are made. The data collection process diagram is shown below Figure 4 shown.
[0053] For subjective data: At the beginning of the experiment, the main tester in the co-pilot seat asked and recorded the subject's baseline motion sickness score. When passing through a typical road condition, he gave driving instructions to the driver and asked and recorded the subject's current motion sickness score. After the task was completed, he asked and recorded the subject's ending baseline motion sickness score.
[0054] For objective physiological data: At the beginning of the experiment, the main examiner on the left side of the second row recorded the data and marked it in the physiological instrument software according to the task point. After the subject answered the subjective score of the starting base state, he sat quietly for one minute. The main examiner marked 0 before and after the meditation. Before passing the typical road scene, the main examiner marked 1. After passing the typical road scene, the main examiner marked 2. After completing the task road, the base state ended. The subject sat quietly for one minute, and the main examiner marked 0. After the meditation was completed, the experiment ended and the main examiner ended the data recording.
[0055] Among them, the in-car camera collects the facial expression data of the passengers: the camera is used to capture the facial expressions of the passengers, such as the opening and closing of the eyes, the movement of the eyebrows, the expressions of the mouth, etc. These expression data can reflect the emotional state of the passengers, such as whether they feel nausea, anxiety or discomfort; the portable physiological signal detection equipment records the physiological signals of the passengers: the physiological signal detection equipment can be a wearable device, a professional physiological data detection equipment or a wearable bracelet, chest strap, etc., which is used to record the physiological indicators of the passengers. These indicators include but are not limited to heart rate, skin resistance, breathing rate, etc. These data can more accurately reflect the physical condition of the passengers, and combined with facial expression data can provide more comprehensive motion sickness assessment information;
[0056] Among them, according to the semantics of the subjective questionnaire MISC, the degree of motion sickness is divided into three levels, 0-1 points means no obvious motion sickness reaction, recorded as motion sickness level 1; 2-5 points means there are symptoms such as sweating, yawning, eyestrain, etc., recorded as motion sickness level 2; when nausea occurs, the MISC score is ≥ 6 points, recorded as motion sickness level 3.
[0057] In addition, two computers were used to record subjective data and objective data respectively. There were slight differences in the collection time of facial expression data and physiological data in objective data. It was necessary to synchronize the data collected between various devices, and UTC was used for data synchronization. UTC, or coordinated universal time, can achieve synchronous and coordinated calculation of time. During the formal experiment, according to the experimental task, the main experimenter marked the start and end of the event on the physiological data software. After unified export, the facial data was checked by itself according to the time to achieve synchronization of the two types of data. Each subject extracted two sections of baseline data at the beginning and end, 30s data before the task started, and 30s data after the task ended, a total of 14 sections of data. There were 40 experimental subjects and 38 valid people. Finally, a total of 532 groups of valid data were obtained.
[0058] In this embodiment, after the facial expression image data is subjected to feature extraction by ResNet (residual neural network) + Lstm (long short-term memory network), a feature vector with a sequence length of 200 is obtained, and the output of Lstm at the last time step is taken as the representative of the entire sequence for feature fusion (the hidden layer of the last time step can retain the key information of the entire input sequence), and after feature weighting with the weight of the motion sickness susceptibility influencing factor output, feature fusion is performed with the physiological data and vehicle driving parameters.
[0059] Each sample of physiological data is a sequence of length 750. After passing through 5 one-dimensional convolutional layers and an adaptive average pooling layer of size 1, it is processed into a feature sequence with 64 channels and length 1. Then, the output dimension is changed from [N, 64, 1] to [N, 64] through dimensionality reduction operation, and the dimension of length 1 is removed to obtain a 64-dimensional feature vector. After feature weighting with the weight of the motion sickness susceptibility influencing factor output, it is used for subsequent feature splicing.
[0060] The sample of each vehicle motion parameter is a sequence of length 155, which includes the vehicle's lateral, longitudinal and vertical speed, acceleration and jerk. After passing through 5 one-dimensional convolutional layers and an adaptive average pooling layer of size 1, it is processed into a feature sequence with 32 channels and a length of 1. Then, the output dimension is changed from [N, 32, 1] to [N, 32] through dimensionality reduction operation, and the dimension of length 1 is removed to obtain a 32-dimensional feature vector. After feature weighting with the weight output by the motion sickness susceptibility influencing factor, it is used for subsequent feature splicing.
[0061] Among them, the weighting mechanism based on the motion sickness susceptibility factor (MSSQ) corresponds to the "neural storage" of the motion sickness generation process to adjust the contribution of features from different data sources. Considering that subjects with different motion sickness susceptibility scores have different sensitivities in facial expressions, physiological states, and reactions to vehicle motion, three weight factors w1, w2, and w3 are calculated through the MSSQ score, corresponding to facial expression features, physiological data features, and vehicle motion parameter features, respectively, and multiplied with the corresponding features before feature fusion to adjust the weight influence of different features in the motion sickness recognition model.
[0062] The passenger's motion sickness susceptibility score (MSSQ) is taken as input and processed by a well-designed multi-layer perceptron (MLP). The MLP consists of two sequentially arranged fully connected layers, which aims to extract the deep features of the motion sickness susceptibility score and map it to a new representation space to obtain three weight factors w1, w2 and w3, which are then activated by a softmax function, which ensures that the three weights are non-negative and sum to 1. In this way, the MLP can not only learn the nonlinear relationship from the motion sickness susceptibility score to the weight factor mapping, but also extract the key factors that affect motion sickness recognition. The motion sickness susceptibility weight output is as follows: Figure 5 shown.
[0063] The characteristic vector corresponding to the physiological data includes physiological characteristic indicators; the physiological characteristic indicators include the mean of electrodermal activation, the maximum of electrodermal activation, the minimum of electrodermal activation, the median of electrodermal activation, the standard deviation of electrodermal activation, the range of electrodermal activation, the skewness of electrodermal activation, the average heart rate of electrocardiogram, the mean of adjacent RR intervals, the standard deviation of adjacent RR intervals, the root mean square difference of adjacent RR intervals, the high-frequency heart rate variability, the percentage of adjacent RR intervals greater than 50ms, the mean of respiratory rate, the mean of respiratory variability, the coefficient of respiratory variability and the standard deviation of consecutive RR intervals.
[0064] Among them, the mean of galvanic skin activation is the average value of galvanic skin response, which describes the overall level of skin electrical activity over a period of time. The maximum value of galvanic skin activation is the maximum value of galvanic skin response, which reflects the peak level of skin electrical activity under certain stimuli or situations. The minimum value of galvanic skin activation is the minimum value of galvanic skin response, which reflects the minimum level of skin electrical activity under certain stimuli or situations. The median of galvanic skin activation is the median of galvanic skin response, which provides the middle level of electrical activity and is not affected by extreme values. The standard deviation of galvanic skin activation is the standard deviation of galvanic skin response, which is the degree of dispersion of response values. The larger the value, the higher the degree of variation. The range of galvanic skin activation is the range of galvanic skin response, which indicates the amplitude range of electrical response over a period of time. The larger the range, the greater the fluctuation. The skewness of galvanic skin activation is the skewness of galvanic skin response, which indicates the degree of asymmetry of the distribution of galvanic skin data. The average heart rate of electrocardiogram is the average heart rate, the number of heart beats per minute. The mean of adjacent RR intervals is the average time interval between two adjacent heartbeats. The standard deviation of adjacent RR intervals is the standard deviation of adjacent heartbeat intervals, which reflects the overall variation of heart rate variability HRV. The root mean square difference of adjacent RR intervals is the average amplitude of the change between adjacent heartbeat intervals, reflecting the sensitive index of the fast-changing component (parasympathetic nerve tension) in HRV. High-frequency heart rate variability is the heart rate variability in the high-frequency band, 0.15 to 0.4Hz, which mainly reflects the activity of the parasympathetic nervous system. The percentage of adjacent RR interval differences greater than 50ms is the percentage of the number of adjacent heartbeat interval differences greater than 50ms in the total number of heartbeats, which is used to evaluate the rhythmicity of the heart. The mean respiratory rate is the average number of breaths in the respiratory cycle, the respiratory frequency. The mean respiratory variability is the time domain index of the average respiratory cycle, which represents the average value of a series of respiratory cycles. The coefficient of respiratory variability is the coefficient of variation based on the respiratory cycle, that is, the ratio of the standard deviation of the respiratory cycle to the mean value, which describes the degree of variation of the respiratory cycle. The standard deviation of continuous RR intervals is the standard deviation of the change in continuous RR intervals in the respiratory cycle. The larger the value, the greater the change.
[0065] In this embodiment, in terms of feature fusion, LSTM can capture the temporal dynamics of the entire sequence when processing facial expression data, wherein the features output by the last time step of LSTM contain the compact information of the entire sequence. In addition, the one-dimensional convolutional neural network extracts features from physiological data and vehicle motion parameters respectively, effectively capturing the local temporal relationships in these data, and then weighting each feature through the weight output of the motion sickness susceptibility factor. Then, these feature vectors from different data sources are concatenated along the feature dimension to form a longer feature vector. This comprehensive feature vector integrates the features of the occupants’ facial expressions, physiological data, and vehicle motion parameters, providing a rich information basis for the prediction of motion sickness levels. The feature fusion process is as follows: Figure 6 shown.
[0066] In this embodiment, Figure 7As shown in the figure, the decision module of the recognition model uses fully connected layers (FC). Fully connected layers are also called multi-layer perceptrons (MLP). In deep learning networks, they mainly play a role in classification. The fused features are passed through the fully connected layers to obtain the output classification results. The process is as follows:
[0067] y=h fc (Concat(w1·S feature ,w2·I feature ,w3·U feature ));
[0068] Among them, h fc is the function of the decision module, S feature ,I feature , U feature They represent facial expression features, physiological signal features and vehicle motion parameter features respectively.
[0069] In this embodiment, the method for assessing the degree of motion sickness of the present invention further includes: performing motion sickness alleviation processing according to the degree of motion sickness of the occupant:
[0070] When the occupants are not in a state of motion sickness, the vehicle driving dynamics parameters are adjusted so that the vehicle driving dynamics parameters are below the mild motion sickness index value; through this processing, on the basis that the occupants are not in a state of motion sickness, the vehicle driving dynamics parameter values are appropriately increased, thereby improving the vehicle operation efficiency.
[0071] When the occupant is in a state of mild motion sickness, the vehicle driving dynamics parameters are adjusted so that the vehicle driving dynamics parameters are below the non-motion sickness index value; through this processing, the vehicle driving dynamics parameter values are appropriately lowered based on the occupant's motion sickness, thereby alleviating the occupant's motion sickness symptoms.
[0072] When the occupant is in a state of severe motion sickness, the vehicle driving dynamics parameters are no longer adjusted, and the vehicle is stopped for rest. In this state, adjusting the driving dynamics parameters may not completely alleviate the occupant's motion sickness symptoms, and stopping for rest is a safer and more effective option.
[0073] The vehicle driving dynamics parameters include the maximum longitudinal acceleration a y _max, longitudinal acceleration change value Δa y And the maximum longitudinal speed V y _max;
[0074] In the curved road condition scenario, the vehicle driving dynamics parameters include the longitudinal acceleration change value Δa y And the mean vertical angular velocity Gyro z_mean.
[0075] The index values corresponding to the vehicle driving dynamics parameters are shown in Table 1:
[0076] Table 1
[0077]
[0078] The present invention also relates to a system for assessing the degree of motion sickness of passengers. The assessment system corresponds to the assessment method of the above embodiment and can be understood as a system for implementing the above assessment. The system includes a collection unit, a feature extraction unit, a feature fusion unit and a motion sickness recognition unit.
[0079] The collection unit is used to collect subjective data and objective data of passengers in different road scenes; wherein the objective data is labeled with the subjective data;
[0080] The feature extraction unit is used to extract features from the annotated objective data and extract vehicle motion parameter feature information;
[0081] The feature fusion unit is used to fuse the extracted feature information to obtain fused feature information;
[0082] The motion sickness recognition unit is used to input the fused feature information into the recognition model and output the motion sickness level recognition result of the occupant.
[0083] It also includes a motion sickness relief unit, which is used to perform motion sickness relief processing according to the motion sickness level of the occupant.
[0084] The present invention provides an objective and scientific method and system for evaluating the degree of motion sickness of passengers, which combines subjective data with objective physiological and facial expression data to comprehensively evaluate the degree of motion sickness of passengers from multiple perspectives. This comprehensive evaluation can more accurately reflect the actual state of motion sickness of passengers, avoid the evaluation bias caused by single data, and provide a more reasonable set of motion sickness evaluation technology. In addition, the driving parameters of the electric vehicle are intelligently adjusted according to the evaluation results, and a personalized motion sickness mitigation control strategy is provided, so as to achieve a balance between passenger comfort and driving efficiency, thereby improving driving safety and efficiency.
[0085] Finally, it should be noted that the above embodiments are only used to illustrate the technical solution of the present invention rather than to limit it. Although the present invention has been described in detail with reference to the preferred embodiments, those skilled in the art should understand that the technical solution of the present invention can be modified or replaced by equivalents without departing from the purpose and scope of the technical solution of the present invention, which should be included in the scope of the claims of the present invention.
Claims
1. A method for assessing the degree of motion sickness of passengers, characterized by: The steps include: S1. Collect subjective data and objective data of passengers in different road scenarios; use subjective data as labels to annotate objective data; S2. Extract features from the labeled objective data and extract feature information of vehicle motion parameters; S3. Fusing the feature information extracted in step S2 to obtain fused feature information; S4. Input the fused feature information into the recognition model and output the recognition result of the occupant's motion sickness level.
2. The method for assessing the degree of motion sickness of passengers according to claim 1, characterized in that: The subjective data include motion sickness level MISC score; the objective data include physiological data and facial expression data.
3. The method for assessing the degree of motion sickness of passengers according to claim 2, characterized in that: Feature extraction is performed on the labeled objective data, including: The facial expression data is subjected to feature extraction through residual neural network and long short-term memory network to obtain a feature vector with a sequence length of m; After the physiological data passes through n one-dimensional convolutional layers and an adaptive average pooling layer of size k, it is processed into a feature sequence with s channels and a length of k. Then, the output dimension is changed from [N, s, 1] to [N, s] through a dimensionality reduction operation, and the dimension of length 1 is removed to obtain an s-dimensional feature vector.
4. The method for assessing the degree of motion sickness of passengers according to claim 1, characterized in that: Extract vehicle motion parameter feature information, including: After passing through n one-dimensional convolutional layers and an adaptive average pooling layer of size k, the vehicle motion parameters are processed into a feature sequence with s / 2 channels and a length of k. Then, the output dimension is changed from [N, s / 2, 1] to [N, s / 2] through a dimensionality reduction operation, and the dimension of length 1 is removed to obtain an s / 2-dimensional feature vector.
5. The method for assessing the degree of motion sickness of passengers according to claim 2, characterized in that: The feature information extracted in step S2 is integrated, specifically including: Weights are configured for physiological data, facial expression data, and vehicle motion parameters respectively, and weighted processing is performed. Then, the feature vectors from different data sources are concatenated along the feature dimension to form a feature vector.
6. The method for assessing the degree of motion sickness of passengers according to claim 3, characterized in that: The characteristic vector corresponding to the physiological data includes physiological characteristic indicators; the physiological characteristic indicators include the mean of electrodermal activation, the maximum of electrodermal activation, the minimum of electrodermal activation, the median of electrodermal activation, the standard deviation of electrodermal activation, the range of electrodermal activation, the skewness of electrodermal activation, the average heart rate of electrocardiogram, the mean of adjacent RR intervals, the standard deviation of adjacent RR intervals, the root mean square difference of adjacent RR intervals, the high-frequency heart rate variability, the percentage of adjacent RR intervals greater than 50ms, the mean of respiratory rate, the mean of respiratory variability, the coefficient of respiratory variability and the standard deviation of consecutive RR intervals.
7. The method for assessing the degree of motion sickness of passengers according to claim 1, characterized in that: Also includes: According to the occupants' motion sickness level, motion sickness relief measures are provided: When the occupant is not in a motion sickness state, the vehicle driving dynamics parameters are adjusted so that the vehicle driving dynamics parameters are below the mild motion sickness index value; When the occupant is in a state of mild motion sickness, the vehicle driving dynamics parameters are adjusted so that the vehicle driving dynamics parameters are below the non-motion sickness index value; When the occupants are in a state of severe motion sickness, the vehicle's driving dynamics parameters are no longer adjusted and the vehicle is stopped for rest.
8. The method for assessing the degree of motion sickness of passengers according to claim 7, characterized in that: In a straight road scenario, the vehicle driving dynamics parameters include the maximum longitudinal acceleration, the longitudinal acceleration change value and the maximum longitudinal driving speed; In a curved road condition scenario, the vehicle driving dynamics parameters include a longitudinal acceleration change value and a vertical angular velocity average value.
9. A system for assessing the degree of motion sickness of passengers, characterized in that: It includes a collection unit, a feature extraction unit, a feature fusion unit and a motion sickness recognition unit; The collection unit is used to collect subjective data and objective data of passengers in different road scenes; wherein the objective data is labeled with the subjective data; The feature extraction unit is used to extract features from the annotated objective data and extract vehicle motion parameter feature information; The feature fusion unit is used to fuse the extracted feature information to obtain fused feature information; The motion sickness recognition unit is used to input the fused feature information into the recognition model and output the motion sickness level recognition result of the occupant.
10. The passenger motion sickness degree assessment system according to claim 9, characterized in that: It also includes a motion sickness relief unit, which is used to perform motion sickness relief processing according to the motion sickness level of the occupant.
Citation Information
Patent Citations
Automobile passenger carsickness grading evaluation model and evaluation method
CN117314010A
Subjective and objective correlation evaluation method for easy carsickness characteristics of drivers and passengers of pure electric drive vehicle type
CN117828493A
Cited By
VR motion sickness state detection method based on multi-modal physiological parameter fusion
CN120392030A
Human body motion sickness grade evaluation method and system for intelligent cockpit
CN120863647A
Motion motion state determination method and device, storage medium and program product
CN122020434A
