A ride comfort evaluation method for intelligent driving vehicles based on multimodal data

Abstract

The present invention belongs to the fields of autonomous driving technology and human-computer interaction, and discloses a method for evaluating the ride comfort of intelligent driving vehicles based on multimodal data. The method uses sensors to collect vehicle dynamic performance data, vehicle driving behavior information, and occupant physiological indicators, while also collecting occupants' subjective scores. The collected data is preprocessed to eliminate invalid physiological reactions, and a subjective score consistency check is performed. After correlation analysis is performed to screen valid data, image conversion is performed to form the final training sample. A systematic autonomous driving vehicle comfort evaluation model is constructed, comprehensively considering the vehicle's dynamic performance, physiological reactions, and subjective feelings. This systematic autonomous driving vehicle ride comfort evaluation model integrates vehicle behavior, occupants' objective reactions, and subjective feelings into the evaluation system, making it more accurate and reliable.

Description

Technical Field

[0001] The present invention belongs to the fields of autonomous driving technology and human-computer interaction, and relates to a method for evaluating ride comfort of an intelligent driving vehicle based on multimodal data. Background Art

[0002] In recent years, with the widespread adoption of electric vehicles and advancements in autonomous driving technology, the role of driver in advanced autonomous vehicles is gradually shifting to passenger. While this shift relieves passengers of the burden of driving, it also increases the risk of discomfort, such as motion sickness. To mitigate this risk, an accurate assessment of passenger comfort is essential.

[0003] However, current evaluations of the comfort of intelligent driving vehicles rely heavily on dynamic indicators such as acceleration or subjective scoring via questionnaires. However, these methods suffer from a lack of physiological signal response modeling, a single data modality, poor scoring consistency, and limited model versatility and intelligence. Therefore, there is an urgent need to develop an intelligent evaluation method that integrates multi-source data and possesses deep perception capabilities to improve the objectivity, accuracy, and real-time nature of comfort analysis.

[0004] Patent application publication number CN112353392B discloses a method for evaluating occupant comfort in intelligent vehicles. The method obtains vital sign information and subjective comfort evaluation indicators of the occupants while the vehicle is stationary and in motion. An objective comfort evaluation model based on the vital sign information is used to calculate an objective comfort evaluation indicator based on the vital sign signals. A vehicle dynamics-based occupant comfort prediction model is used to obtain a vehicle dynamics-based occupant comfort prediction evaluation indicator. Finally, a comprehensive occupant comfort evaluation model is constructed. Based on the comprehensive occupant comfort evaluation indicator predicted by the comprehensive occupant comfort evaluation model and the vehicle's three-degree-of-freedom model, a vehicle dynamics control domain based on occupant comfort is established to ensure ride comfort. However, this method tends to be deployed in applications. Its modeling approach for ride comfort evaluation is traditional, with limited ability to express complex nonlinear characteristics. The output is a comprehensive indicator K, which lacks fine-grained scoring differentiation.

[0005] Based on the above background, it can be seen that in order to overcome the shortcomings of existing evaluation methods, it is necessary to propose a new evaluation method for the ride comfort of autonomous driving vehicles to improve the evaluation accuracy and practicality. Summary of the Invention

[0006] In order to solve the above technical problems, the present invention provides a method for evaluating the ride comfort of an intelligent driving vehicle based on multimodal data.

[0007] The technical solution of the present invention:

[0008] A method for evaluating ride comfort of an intelligent driving vehicle based on multimodal data, comprising the following steps:

[0009] S1. Collect target data;

[0010] S11. Determine test conditions: Clarify the test environment and conditions, including vehicle type, road conditions, and driving speed;

[0011] S12. Arrange sensors: Arrange sensors on the vehicle and the human body, including a triaxial accelerometer, skin galvanometer electrodes and a skin galvanometer, and electrocardiogram electrodes and an electrocardiogram. Connect the computer to the vehicle's OBD-II interface via a USB-CAN interface device. The triaxial accelerometer is used to collect vehicle dynamic performance data. The computer is connected to the vehicle's OBD-II interface via a USB-CAN interface device to read driving behavior parameters from the CAN bus. The skin galvanometer electrodes and the skin galvanometer, and the electrocardiogram electrodes and the electrocardiogram are used to collect physiological indicators of the human body.

[0012] The vehicle's dynamic performance data is collected by a three-axis acceleration sensor installed under the passenger seat. The three-axis acceleration sensor collects dynamic acceleration along the vehicle's forward direction (X-axis), the horizontal Y-axis perpendicular to the forward direction on the horizontal plane, and the vertical Z-axis perpendicular to the forward direction on the vertical plane.

[0013] To collect driving behavior parameters, the computer is connected to the vehicle's OBD-II interface via a USB-CAN interface device and reads them via the CAN bus. Driving behavior parameters include vehicle speed, engine speed, throttle opening, accelerator pedal opening, and lane changing, acceleration, and overtaking driving operations, which are manually recorded and synchronously annotated with timestamps. The duration of the operation is calculated using the start and end timestamps of the action.

[0014] To collect physiological indicators of the human body, the skin electrode collection electrodes are fixed on the surface of the thenar muscles of both palms to measure the skin electrode signals of the human body and are connected to the skin electrode meter via wireless transmission equipment; the electrocardiogram collection electrodes are arranged on the chest, left arm, and right arm to measure the electrocardiogram signals of the human body and are connected to the electrocardiogram meter via wireless transmission equipment; the skin electrode meter and the electrocardiogram meter are connected to the computer through their microcomputer interfaces to transmit the collected signals to the computer; among them, the electrocardiogram collection electrodes are three-lead patch electrodes for electrocardiogram collection;

[0015] S13. Collect required information: Vehicle dynamic performance data and driving behavior parameters under different conditions recorded by the vehicle's OBD are used to evaluate vehicle dynamic performance and its impact on comfort. Simultaneously record human physiological indicators to quantify the human body's physiological response to vehicle motion. Collect subjective ratings from passengers or drivers to evaluate comfort based on their actual experience. The rating range is [1, 10], with a 0.5-point interval, where 1 is extremely uncomfortable and 10 is very comfortable.

[0016] S2. Filter valid data through correlation analysis;

[0017] S21. Data preprocessing: The raw data collected in step S1 are systematically preprocessed. First, the subjective scores are filtered by intervals. Second, the physiological response amplitude is judged. If the standard deviation of the heart rate variability is less than 2 bpm or the rate of change of the electrodermal amplitude is less than 5%, it is considered an invalid response and the data is discarded.

[0018] The preprocessing method is as follows: the raw data collected in step S1 are systematically preprocessed. First, the subjective scores are filtered to retain the subjective scores between [3, 8]. Second, the physiological response amplitude is judged. The ECG signal is the standard deviation of the heart rate variability (SDNN) in the 5-second window before and after the manually recorded lane changing, acceleration, and overtaking driving operations:

[0019]

[0020] Where n is the total number of RR intervals, is the i-th RR interval, is the average value of all RR intervals in the window; where the RR interval is the time interval between an R wave in an electrocardiogram and the next R wave in the window;

[0021] If the standard deviation of heart rate variability SDNN is less than 2 bpm, it is considered an invalid response and the corresponding data are discarded;

[0022] The skin electrical signal is calculated by calculating the skin electrical amplitude change rate of the manually recorded 5-second window before and 10-second window after the lane changing, acceleration, and overtaking driving operations:

[0023] First, calculate the average skin conductance values in the 5s window before and 10s window after the driving operation:

[0024]

[0025] Where N is the number of sampling points in the window, is the skin conductance value at the ath moment;

[0026] The rate of change of skin electrical amplitude is:

[0027]

[0028] in, is the average skin conductance value in the 10s window after the operation, is the average skin conductance value in the 5-s window before the operation;

[0029] like <5% was considered as an invalid reaction and the corresponding data were excluded;

[0030] S22. Consistency Verification of Subjective Scores: Use the consistency analysis coefficient to analyze the preprocessed data, eliminating samples where multiple evaluators have significant differences in their ratings for the same driving maneuver, as well as samples where a single evaluator has significant differences in their ratings before and after the same driving maneuver. Perform segmented balancing on the subjective scores to avoid sample bias in the training set.

[0031] The method for checking the consistency of subjective ratings is to use Kendall's W coordination coefficient to analyze the coordination of multiple evaluators' ratings on the same driving operation. First, calculate the total sum of squares of rank differences. :

[0032]

[0033] in, is the sum of the ranks of the bth sample, that is, the sum of the scores of all evaluators on the sample. is the average of the sum of all sample ranks;

[0034] Then Kendall's W coordination coefficient is:

[0035]

[0036] in, is the number of evaluators, The number of samples evaluated by each evaluator, is the total sum of squares of rank differences;

[0037] like <0.7, it is considered that multiple evaluators have significant differences in their ratings for the same driving operation, and the corresponding data are eliminated;

[0038] Cronbach's α internal consistency coefficient was used to measure the consistency of a single evaluator's ratings on the same driving operation multiple times:

[0039]

[0040] in, is the number of items, i.e., the number of identical driving maneuvers scored by a single evaluator. is the variance of the score of the c-th identical driving operation, is the overall variance of the evaluators' ratings in all the same driving operations;

[0041] like If the score is less than 0.75, it is considered that the individual evaluator's scores for the same driving operation under multiple conditions have large differences before and after, and the corresponding data are eliminated;

[0042] For the subjective rating y∈[3, 8] segment, the segmented random downsampling is used to balance the sample size. The subjective rating is divided into several sub-intervals. The number of samples in the rating segment with the least samples is used as the upper limit. The remaining rating segments are randomly downsampled to this number to balance the samples of low, medium and high rating segments.

[0043] S23. Data Image Conversion: Vehicle dynamic performance data and human physiological indicator data are extracted through time windows and subjected to time-frequency conversion to generate frequency domain images, which include acceleration images and physiological images. Driving behavior parameters are used to construct a structured driving behavior vector. The subjective score serves as a label for the subsequent training target value of a convolutional neural network-based autonomous driving vehicle ride comfort evaluation model. The acceleration image, physiological image, and structured driving behavior vector serve as inputs to the convolutional neural network-based autonomous driving vehicle ride comfort evaluation model. A training set is constructed using the acceleration image, physiological image, and structured driving behavior vector and is labeled with the subjective score. Each sample consists of {acceleration image, physiological image, structured driving behavior vector, and subjective score label}.

[0044] After screening, the sample data retained by S22 is converted into an image. The dynamic acceleration of the vehicle's dynamic performance data on the X, Y, and Z axes is converted into a Mel spectrum through short-time Fourier transform and Mel filtering. The Mel spectrum of the X, Y, and Z axes is mapped to R, G, and B channels respectively, and combined into a three-channel color image to form an acceleration image. The electrocardiogram and electrodermal signals of the human body's physiological indicator data are converted into spectrograms using CWT and Mel spectrum respectively to form a physiological image. The driving behavior parameters are not converted into images, and a structured driving behavior vector is constructed. The acceleration image and physiological image are unified in size to 224×224×3. All inputs are bound to the score labels to form the final training sample:

[0045] ;

[0046] in, is the acceleration image, For physiological images, is the structured driving behavior vector, is the subjective rating label value;

[0047] S3. Constructing a ride comfort evaluation model for autonomous vehicles based on a convolutional neural network: Inputting three-channel information, namely, vehicle dynamic performance data, human physiological indicators, and driving behavior parameters, into the constructed ride comfort evaluation model for autonomous vehicles based on a convolutional neural network;

[0048] The ride comfort evaluation model of autonomous driving vehicles based on convolutional neural networks is:

[0049] Build a three-way neural network system:

[0050] Convolutional neural network 1 branch extracts acceleration image features: ;

[0051] The convolutional neural network branch 2 extracts the image features of the electrocardiogram signal and the electrodermal signal: ;

[0052] The multi-layer perceptron branch processes the driving behavior vector: ;

[0053] The output features of the three-way neural network system are fused and then nonlinearly mapped:

[0054] ;

[0055] in, ;

[0056] ;

[0057] in, is the first layer fully connected weight matrix; is the first layer bias term; ReLU(x)=max(0,x) is the activation function of the rectified linear unit. is the output weight of the second layer; is the output layer bias term; The comfort score prediction value output by the autonomous driving vehicle ride comfort evaluation model;

[0058] Train the network with mean squared error loss function:

[0059] ;

[0060] in, is the total number of samples in the training set, is the true subjective rating label value of the i-th sample, is the score value predicted by the i-th sample model, The target loss for model training is the mean square error loss;

[0061] S4. Output of comfort evaluation results.

[0062] The beneficial effects of the present invention are as follows: the acceleration information and physiological index information used in the present invention are input as images, which have strong visualization, and the driving behavior information is input as a behavior vector, which is convenient for interpretation and visual tuning; the deep model used in the present invention has strong expressive power, especially high ability to capture nonlinear physiological changes and adaptive behavior characteristics; the comfort evaluation method of the present invention is not limited to certain working conditions, and any working condition can be evaluated using this method as long as relevant driving behavior information input is collected. BRIEF DESCRIPTION OF THE DRAWINGS

[0063] Figure 1 This is an overall flow chart of a method for evaluating ride comfort of an intelligent driving vehicle based on multimodal data according to the present invention;

[0064] Figure 2 This is a flow chart of an evaluation model for a method for evaluating ride comfort of an intelligent driving vehicle based on multimodal data according to the present invention;

[0065] Figure 3 It is a multimodal evaluation index of the intelligent driving vehicle ride comfort evaluation method based on multimodal data of the present invention. DETAILED DESCRIPTION

[0066] The following further illustrates the specific implementation of the present invention in conjunction with the accompanying drawings and technical solutions.

[0067] like Figure 1 As shown, this embodiment exemplarily shows a method for evaluating ride comfort of an intelligent driving vehicle based on multimodal data, comprising the following steps:

[0068] 1) Determine the test conditions: First, the vehicle type should have typical intelligent driving capabilities and stable power response characteristics. Secondly, the test road environment is selected to include mixed traffic scenarios of urban main roads and expressways, with a variety of typical operating conditions such as lane changing, acceleration, braking, ramps, etc., to ensure that the test covers a variety of driving conditions. In terms of driving speed setting, in order to avoid the weakening of the operating impact at too low speed or uncontrollable data fluctuations at high speed, the test limits the vehicle to operate within the range of 30-80km / h and monitors the speed in real time through GPS. In each set scenario, the operation trigger time point is calibrated by preset signs, voice prompts or manual recording by the tester to ensure the consistency and timing alignment of behavioral events and collected signals. In addition, in order to avoid environmental factors interfering with the physiological state of the occupants, the entire experimental process is carried out under relatively constant outdoor temperature and humidity conditions, and the temperature in the car is adjusted between 22℃±1℃ to avoid errors in the skin's electrical response caused by hot and cold stimuli;

[0069] 2) Sensor Placement: Vehicle dynamic performance data is collected using a triaxial accelerometer. The sensor is secured to the bottom of the passenger seat using double-sided tape and anti-vibration brackets, ensuring close contact with the rigid connection points of the vehicle structure to prevent interference with vibration signals from the seat's flexible structure. Occupant physiological indicators are collected using physiological signal acquisition equipment such as the BIOPACMP36 multi-channel physiological signal acquisition system. Electrocardiogram (ECG) signals are collected using the ECG100C module, with three-lead electrodes attached to the chest, left arm, and right arm. Skin conductance signals are collected using the EDA100C module, with electrodes attached to the thenar muscles of both hands to improve conductivity and reduce motion artifacts.

[0070] To ensure stable electrode adhesion and excellent signal channel quality, all test passengers were required to perform a maximum voluntary contraction test before the formal test. This test involves the subject actively moving both arms slightly, then simultaneously opening and closing their palms, to simulate potential displacement interference during the experiment. During this time, the instrument synchronously records changes in skin conductance and ECG signals, detecting any electrode detachment, signal loss, severe noise, or artifacts. This process also records the peak skin conductance response at maximum tension, which serves as a reference for the individual physiological response amplitude and facilitates subsequent data normalization and response sensitivity analysis.

[0071] 3) Collecting required data: First, the computer uses the vehicle's OBD interface to access the CAN bus and uses a Bluetooth adapter to record multiple driving behavior parameters in real time during vehicle operation, including vehicle speed, throttle opening, braking status, engine speed, etc. In typical operating scenarios such as lane changing, acceleration, and overtaking, the computer also extracts the vehicle's dynamic acceleration along the X-axis of the forward direction, the Y-axis perpendicular to the forward direction on the horizontal plane, and the Z-axis perpendicular to the forward direction on the vertical plane.

[0072] Secondly, the ECG and galvanic skin signals were recorded synchronously during the acquisition test. A fixed time window was set before and after each driving operation. The ECG extraction window was 5 seconds before and after the driving operation, and the galvanic skin extraction window was 5 seconds before and 10 seconds after the driving operation.

[0073] Finally, after each driving maneuver, occupants were asked to subjectively rate their experience based on their true feelings. The rating scale used an equally spaced scale of 1 to 10, with 1 indicating extreme discomfort and 10 indicating complete comfort, with 0.5-point intervals added. The rating was completed by the occupants using a tablet interface or voice input, and was automatically linked to the maneuver number, timestamp, and corresponding multimodal data. The subjective comfort evaluation table is shown in Table 1.

[0074] Table 1 Subjective evaluation table of comfort 4)

[0076] 5) Correlation analysis to screen valid data:

[0077] Data preprocessing: The collected raw data were systematically preprocessed. First, the subjective score interval was filtered, and subjective scores between [3 and 8] were retained. Second, the physiological response amplitude was judged and the corresponding physiological waveform segments were extracted. The ECG signal was used to calculate the standard deviation of heart rate variability (SDNN) within a 5-second window before and after the manually recorded lane changing, acceleration, and overtaking driving operations:

[0078]

[0079] Where n is the total number of RR intervals, is the i-th RR interval, is the average value of all RR intervals in the window; where the RR interval is the time interval between an R wave in an electrocardiogram and the next R wave in the window;

[0080] If the standard deviation of heart rate variability SDNN is less than 2 bpm, it is considered an invalid response and the corresponding data are discarded;

[0081] The skin electrical signal is calculated by calculating the skin electrical amplitude change rate of the manually recorded 5-second window before and 10-second window after the lane changing, acceleration, and overtaking driving operations:

[0082] First, calculate the average skin conductance values in the 5s window before and 10s window after the driving operation:

[0083]

[0084] Where N is the number of sampling points in the window, is the skin conductance value at the ath moment;

[0085] The rate of change of skin electrical amplitude is:

[0086]

[0087] in, is the average skin conductance value in the 10s window after the operation, is the average skin conductance value in the 5-s window before the operation;

[0088] like If the reaction rate is less than 5%, it is considered as an invalid reaction and the corresponding data are excluded.

[0089] Consistency check of subjective ratings: Kendall's W coordination coefficient is used to analyze the coordination of multiple evaluators' ratings on the same driving operation. First, the total sum of squares of rank differences is calculated. :

[0090]

[0091] in, is the sum of the ranks of the bth sample, that is, the sum of the scores of all evaluators on the sample. is the average of the sum of all sample ranks;

[0092] Then Kendall's W coordination coefficient is:

[0093]

[0094] in, is the number of evaluators, The number of samples evaluated by each evaluator, is the total sum of squares of rank differences;

[0095] like <0.7, it is considered that multiple evaluators have significant differences in their ratings for the same driving operation, and the corresponding data are eliminated;

[0096] Cronbach's α internal consistency coefficient was used to measure the consistency of a single evaluator's ratings on the same driving operation multiple times:

[0097]

[0098] in, is the number of items, i.e., the number of identical driving maneuvers scored by a single evaluator. is the variance of the score of the c-th identical driving operation, is the overall variance of the evaluators' ratings in all the same driving operations;

[0099] like If the score is less than 0.75, it is considered that the individual evaluator's scores for the same driving operation under multiple conditions have large differences before and after, and the corresponding data are eliminated;

[0100] After removing outlier samples, the original rating labels exhibit a significant skewed distribution, with samples highly concentrated in the medium rating range. This poses a risk of predictive bias during model training. To ensure that samples from all rating ranges are fully learned during training, this paper divides the rating labels into three segments: low [3, 4.5], medium [5, 6.5], and high [7, 8]. The number of samples in the segment with the fewest samples is used as the upper limit, and the remaining rating segments are randomly downsampled to this number to balance the samples in the low, medium, and high rating ranges, thereby constructing a balanced training set.

[0101] Data image conversion: The sample data retained after screening is converted to an image. The dynamic acceleration of the vehicle's dynamic performance data on the X, Y, and Z axes is converted into a mel-spectrogram using short-time Fourier transform and mel filtering. The mel-spectrograms of the X, Y, and Z axes are mapped to R, G, and B channels, respectively, and combined into a three-channel color image to form an acceleration image. The electrocardiogram (ECG) and electrodermal signals of the occupant's physiological indicators are converted to spectrograms using CWT and mel-spectrogram to form a physiological image. Driving behavior parameters are not converted into images, but a structured driving behavior vector is constructed. The acceleration image and physiological image are unified in size to 224×224×3. All inputs are bound to subjective score labels to form the final training sample:

[0102] ;

[0103] in, is the acceleration image, For physiological images, is the structured driving behavior vector, is the subjective rating label value.

[0104] 6) Constructing a ride comfort evaluation model for autonomous vehicles based on convolutional neural networks:

[0105] Build a three-way neural network system:

[0106] Convolutional neural network 1 branch extracts acceleration image features: ;

[0107] Convolutional neural network 2 branches extract physiological image features: ;

[0108] The multi-layer perceptron branch processes the driving behavior vector: ;

[0109] The three-way output features are fused and then nonlinearly mapped:

[0110] ;

[0111] in,

[0112] ;

[0113] in, is the first layer fully connected weight matrix, is the first layer bias term, is the rectified linear unit activation function: ReLU(x)=max(0,x), is the output weight of the second layer, is the output layer bias term, The predicted value of the comfort score output by the model.

[0114] Train the network with mean squared error loss function:

[0115] ;

[0116] in, is the total number of samples in the training set, is the true subjective rating value label of the i-th sample, is the score value predicted by the i-th sample model, The target loss for model training is the mean square error loss.

[0117] Finally, the evaluation results of the ride comfort of intelligent driving vehicles are output; Figure 2 This is a flow chart of an evaluation model for a method for evaluating ride comfort of an intelligent driving vehicle based on multimodal data according to the present invention. Figure 3It is a multimodal evaluation index of the intelligent driving vehicle ride comfort evaluation method based on multimodal data of the present invention.

[0118] The technical solution of the present invention is not limited to the above-mentioned specific embodiments. Any technical variations made according to the technical solution of the present invention fall within the protection scope of the present invention.

Claims

1. A method for evaluating ride comfort of an intelligent driving vehicle based on multimodal data, characterized in that: Here are the steps: S1. Collect target data; S11. Determine test conditions: Clarify the test environment and conditions, including vehicle type, road conditions, and driving speed; S12. Arrange sensors: Arrange sensors on the vehicle and on the body of the occupants. The sensors include a three-axis acceleration sensor, skin electrical collection electrodes and skin electrical meter, electrocardiogram collection electrodes and electrocardiogram. The computer uses Interface device connected to vehicle Interface; Among them, the three-axis acceleration sensor is used to collect vehicle dynamic performance data, and the computer uses Interface device connected to vehicle Interface reading Driving behavior parameters in the bus, skin electrode collection electrodes and skin electrode meter, electrocardiogram collection electrodes and electrocardiogram are used to collect physiological indicators of the human body; S13. Collecting required information: Through vehicle The vehicle dynamic performance data and driving behavior parameters recorded under different conditions are used to evaluate the vehicle dynamic performance and its impact on comfort; the physiological indicators of the occupants are recorded simultaneously to quantify the physiological response of the human body to the vehicle movement; the subjective scores of the occupants are collected and the comfort is evaluated based on their actual feelings. The evaluation range is [1,10] points, and the Divide into intervals, Divided into extremely uncomfortable, Very comfortable; S2. Filter valid data through correlation analysis; S21. Data preprocessing: The raw data collected in step S1 are systematically preprocessed. First, the subjective scores are filtered by intervals. Second, the physiological response amplitude is judged. If the standard deviation of the heart rate variability is less than 2 bpm or the skin conductance amplitude change rate is less than 5%, it is considered an invalid response and the data is discarded. S22. Consistency Verification of Subjective Scores: Use the consistency analysis coefficient to analyze the preprocessed data, eliminating samples where multiple evaluators have significant differences in their ratings for the same driving maneuver, as well as samples where a single evaluator has significant differences in their ratings before and after the same driving maneuver. Perform segmented balancing on the subjective scores to avoid sample bias in the training set. S23. Data Image Conversion: Vehicle dynamic performance data and occupant physiological indicator data are extracted through time windows and subjected to time-frequency conversion to generate frequency domain images, which include acceleration images and physiological images. Driving behavior parameters are used to construct a structured driving behavior vector. The subjective score serves as a label for the subsequent training target value of a convolutional neural network-based autonomous driving vehicle ride comfort evaluation model. The acceleration image, physiological image, and structured driving behavior vector serve as inputs to the convolutional neural network-based autonomous driving vehicle ride comfort evaluation model. A training set is constructed using the acceleration image, physiological image, and structured driving behavior vector and is labeled with the subjective score. Each sample consists of {acceleration image, physiological image, structured driving behavior vector, and subjective score label}. After screening, the sample data retained by S22 is converted into an image. The dynamic acceleration of the vehicle's dynamic performance data on the X, Y, and Z axes is converted into a Mel spectrum through short-time Fourier transform and Mel filtering. The Mel spectrum of the X, Y, and Z axes is mapped to R, G, and B channels respectively, and combined into a three-channel color image to form an acceleration image. The electrocardiogram and electrodermal signals of the occupant's physiological index data are converted into spectrograms using CWT and Mel spectrum respectively to form a physiological image. The driving behavior parameters are not converted into images, and a structured driving behavior vector is constructed. The acceleration image and physiological image are unified in size to 224×224×3. All inputs are bound to subjective score labels to form the final training sample: ； in, is the acceleration image, For physiological images, is the structured driving behavior vector, is the subjective rating label value; S3. Constructing a ride comfort evaluation model for autonomous vehicles based on a convolutional neural network: Inputting three-channel information, namely, vehicle dynamic performance data, occupant physiological indicators, and driving behavior parameters, into the constructed ride comfort evaluation model for autonomous vehicles based on a convolutional neural network; S4. Output of comfort evaluation results.

2. The method for evaluating ride comfort of an intelligent driving vehicle based on multimodal data according to claim 1, characterized in that: In step S12, The vehicle's dynamic performance data is collected by a three-axis acceleration sensor installed under the passenger seat. The three-axis acceleration sensor collects dynamic acceleration along the vehicle's forward direction (X-axis), the horizontal Y-axis perpendicular to the forward direction on the horizontal plane, and the vertical Z-axis perpendicular to the forward direction on the vertical plane. Collect driving behavior parameters, and the computer Interface device connected to vehicle interface, through Bus reading; driving behavior parameters include vehicle speed, engine speed, throttle opening, accelerator pedal opening, and lane changing, acceleration, and overtaking driving operations that are manually recorded and synchronously marked with timestamps. The duration of the operation is calculated based on the start and end timestamps of the action; To collect physiological indicators of the occupants, skin electrode collection electrodes are fixed on the surface of thenar muscles of both palms to measure the skin electrode signals of the occupants and are connected to the skin electrode meter via wireless transmission equipment; ECG collection electrodes are arranged on the chest, left arm and right arm to measure the ECG signals of the occupants and are connected to the ECG meter via wireless transmission equipment; The skin electrode and electrocardiogram are connected to the computer through their microcomputer interfaces, and the collected skin electrode signals and electrocardiogram signals are transmitted to the computer; wherein, the electrocardiogram collection electrodes are three-lead patch electrodes for electrocardiogram collection.

3. The method for evaluating ride comfort of an intelligent driving vehicle based on multimodal data according to claim 2, characterized in that: In step S21, The preprocessing method is as follows: the raw data collected in step S1 are systematically preprocessed. First, the subjective scores are filtered to retain the subjective scores between [3, 8] points. Secondly, the physiological response amplitude is judged. The ECG signals are calculated before and after the manually recorded lane changing, acceleration and overtaking driving operations. Standard deviation of heart rate variability within the window SDNN: ,in, for The total number of intervals, For the i-th Interphase, For all The average value of the interval; The interval is the number of seconds in an electrocardiogram within the window. Wave to the next the time interval between waves; If the standard deviation of heart rate variability , considered as invalid response, and the corresponding data were eliminated; Skin electrical signals are calculated for manually recorded lane changing, acceleration and overtaking driving operations. After window and driving operation The rate of change of skin electrical amplitude of the window: First, calculate the driving operation After window and driving operation Average skin conductance value within the window: ,in, is the number of sampling points in the window, For the Skin conductance value at the moment; The rate of change of skin electrical amplitude is: ,in, After the operation The average skin conductance value of the window, Before operation Average skin conductance value of the window; like , considered as invalid response, and the corresponding data were eliminated.

4. The method for evaluating ride comfort of an intelligent driving vehicle based on multimodal data according to claim 3, characterized in that: In step S22, The method for checking the consistency of subjective scores is to use Kendall's The coordination coefficient analyzes the coordination of multiple evaluators' scores on the same driving operation. First, the total sum of squares of the rank differences is calculated. : ,in, For the The sum of the ranks of a sample is the sum of the scores of all evaluators on the sample. is the average of the sum of all sample ranks; Kendall's The coordination coefficient is: ,in, is the number of evaluators, The number of samples evaluated by each evaluator, is the total sum of squares of rank differences; like , if multiple evaluators have significant differences in their ratings for the same driving operation, the corresponding data will be eliminated; Cronbach's α internal consistency coefficient was used to measure the consistency of a single evaluator's ratings on the same driving operation multiple times: ,in, is the number of items, i.e., the number of identical driving maneuvers scored by a single evaluator. For the The variance of the same driving operation score, is the overall variance of the evaluators' ratings in all identical driving operations; like If the score is less than 0.75, it is considered that the individual evaluator's scores for the same driving operation under multiple conditions have large differences before and after, and the corresponding data are eliminated; Subjective rating Segmentation: Use segmented random downsampling to balance the number of samples, divide the subjective ratings into several sub-intervals, take the number of samples in the rating segment with the least samples as the upper limit, and randomly downsample the remaining rating segments to this number to balance the samples in the low, medium, and high rating segments.

5. The method for evaluating ride comfort of an intelligent driving vehicle based on multimodal data according to claim 4, characterized in that: In step S3, The ride comfort evaluation model of autonomous driving vehicles based on convolutional neural networks is: Build a three-way neural network system: Convolutional neural network 1 branch extracts acceleration image features: ; Convolutional neural network 2 branches extract physiological image features: ; The multi-layer perceptron branch processes the driving behavior vector: ; The output features of the three-way neural network system are fused and then nonlinearly mapped: ； in, ; ； in, is the first layer fully connected weight matrix; is the first layer bias term; ReLU(x)=max(0,x) is the activation function of the rectified linear unit. is the output weight of the second layer; is the output layer bias term; The comfort score prediction value output by the autonomous driving vehicle ride comfort evaluation model; The mean square error loss function is used to train the autonomous driving vehicle ride comfort evaluation model: ； in, is the total number of samples in the training set, is the true subjective rating label value of the i-th sample, is the score value predicted by the i-th sample model, The target loss for model training is the mean square error loss.

Images