Automatic evaluation method for driving behavior of urban rail driver based on simulated driving scene

Abstract

The application relates to an automatic evaluation method for urban rail driver driving behavior based on a simulated driving scene, which comprises the following steps: collecting multi-source signals under a standard driving operation process, aligning the multi-source signals after time sequence coding, voice feature extraction and trajectory recording, and generating a multi-modal standard behavior template library; synchronously collecting four types of data of operation behavior, voice features, vehicle dynamics and video images, aligning and fusing the four types of data according to time stamps, and generating a multi-modal driving behavior data set; comparing the data set with the standard template library item by item, calculating behavior deviation quantitative values in each dimension and generating a multi-dimensional deviation feature set; according to the multi-dimensional deviation feature set, calculating each normative score respectively, then performing weighted fusion according to preset weights, and generating a driving behavior basic score; based on video image data and vehicle dynamic time sequence data in the data set, identifying fatigue driving behavior and aggressive driving behavior respectively, performing score deduction processing on the basic score according to the identification results, and generating a driving behavior comprehensive evaluation result.

Description

Technical Field

[0001] This invention relates to the field of intelligent assessment technology for urban rail transit, specifically to an automatic assessment method for urban rail transit driver behavior based on simulated driving scenarios. Background Technology

[0002] As the backbone of urban public transportation, urban rail transit's operational safety heavily relies on the drivers' adherence to standardized operating procedures, emergency response capabilities, and sustained vigilance. The automated assessment method for urban rail transit driver behavior refers to a technical means of quantitatively evaluating the compliance and safety of driver behavior by automatically analyzing and judging this data based on standardized operating procedures during simulated driving training or assessment scenarios. This method plays a crucial supporting role in urban rail transit driver vocational skills assessment, pre-employment training assessment, and annual retraining evaluation. The objectivity and accuracy of its assessment results directly affect the fairness of driver qualification certification and the level of safety assurance for line operations.

[0003] Currently, the evaluation of urban rail drivers' driving behavior generally relies on examiners to conduct manual observation and subjective scoring at the simulated driving test site, lacking a systematic and automated evaluation method. Specifically, existing technologies suffer from the following main shortcomings: First, evaluation standards are difficult to standardize objectively. Examiners may apply different scoring criteria to different batches and candidates, leading to significant differences in scores for the same operational deviation under different examiners' judgments, resulting in insufficient consistency and repeatability of the evaluation. Second, in existing solutions, multi-source data such as operational actions, voice control, and vehicle dynamic response are recorded in their respective independent acquisition channels, lacking an effective time synchronization and data fusion mechanism. This makes it impossible to perform precise time correlation analysis between operational behavior and corresponding vehicle dynamic response, making it difficult to trace deductions back to specific driving moments and operational events. Third, the identification of abnormal driving behaviors (such as sluggish operation under fatigue or abnormal speed fluctuations caused by aggressive driving) in the existing evaluation process mainly relies on the examiner's real-time observation, lacking a real-time detection and automatic warning mechanism based on multimodal data. This makes it difficult to promptly detect and record sudden risky driving behaviors during the assessment process, and there is a lack of complete operational data backtracking methods after the assessment. When candidates raise objections to their scores, they cannot provide objective operational records as a basis for review. Summary of the Invention

[0004] Based on this, the purpose of the present invention is to provide an automatic evaluation method for urban rail transit driver behavior in a simulated driving scenario that can integrate multi-source synchronous data and achieve quantitative evaluation.

[0005] The objective of this invention is achieved through the following solution:

[0006] In a first aspect, the present invention provides an automatic evaluation method for urban rail transit driver behavior based on simulated driving scenarios, comprising the following steps:

[0007] S1: Collect the operation signals, voice signals and vehicle dynamic signals of urban rail drivers under the standard driving operation procedure through the simulated driving assessment equipment. After performing time-series coding, voice feature extraction and trajectory recording on the operation signals, voice signals and vehicle dynamic signals respectively, align them with a unified time reference to generate a multimodal standard behavior template library.

[0008] S2: Collect time-series data of the examinee's operational behavior, speech features, vehicle dynamics, and video images during the assessment process at a preset sampling period, and then merge them after aligning them by timestamp to generate a multimodal driving behavior dataset;

[0009] S3: Compare the time-series data of operation behavior, speech feature, and vehicle dynamics in the multimodal driving behavior dataset with the multimodal standard behavior template library item by item, calculate the behavior deviation quantification value under each dimension, and generate a multi-dimensional deviation feature set containing operation deviation features, speech deviation features, and dynamic deviation features.

[0010] S4: Calculate the operation standardization score, voice standardization score and driving stability score respectively based on the multi-dimensional deviation feature set, and then weight and fuse the operation standardization score, voice standardization score and driving stability score according to the preset weight to generate a basic driving behavior score;

[0011] S5: Based on video image data in the multimodal driving behavior dataset, extract the coordinates of key points on the driver's face and calculate the time series of eyelid opening and closing. Simultaneously, calculate the rate of change of speed and the rate of change of acceleration based on vehicle dynamic time series data. Compare the time series of eyelid opening and closing with a preset fatigue threshold to identify fatigued driving behavior, and compare the rate of change of speed and the rate of change of acceleration with a preset aggressive threshold to identify aggressive driving behavior. When abnormal driving behavior is identified, generate an abnormal behavior warning log containing the abnormal type and corresponding deduction value. Based on the abnormal behavior warning log, accumulate deductions are made on the basic driving behavior score to generate a comprehensive driving behavior evaluation result.

[0012] Secondly, this invention provides an automatic evaluation system for urban rail transit driver behavior based on simulated driving scenarios, the system being configured with the following modules:

[0013] The standard behavior template construction module is used to collect the operation signals, voice signals and vehicle dynamic signals of urban rail drivers under the standard driving operation process through the simulated driving assessment equipment. After performing time-series encoding, voice feature extraction and trajectory recording on the operation signals, voice signals and vehicle dynamic signals respectively, they are aligned with a unified time reference to generate a multimodal standard behavior template library.

[0014] The candidate multimodal data fusion module is used to collect time-series data of candidates' operational behavior, speech features, vehicle dynamics, and video images during the assessment process at a preset sampling period, and then fuse them after aligning them by timestamp to generate a multimodal driving behavior dataset.

[0015] The multi-dimensional behavior deviation calculation module is used to compare the operation behavior time-series data, voice feature time-series data, and vehicle dynamic time-series data in the multimodal driving behavior dataset with the multimodal standard behavior template library item by item, calculate the behavior deviation quantification value under each dimension, and generate a multi-dimensional deviation feature set containing operation deviation features, voice deviation features, and dynamic deviation features.

[0016] The driving basic score fusion module is used to calculate the operation standardization score, voice standardization score and driving stability score respectively based on the multi-dimensional deviation feature set, and then weight and fuse the operation standardization score, voice standardization score and driving stability score according to preset weights to generate a driving behavior basic score.

[0017] The anomaly identification and comprehensive evaluation module is used to extract the coordinates of key facial points of the driver based on video image data in the multimodal driving behavior dataset and calculate the time series of eyelid opening and closing. At the same time, it calculates the rate of change of speed and the rate of change of acceleration based on vehicle dynamic time series data. The eyelid opening and closing time series is compared with a preset fatigue threshold to identify fatigue driving behavior, and the rate of change of speed and the rate of change of acceleration are compared with a preset aggressive threshold to identify aggressive driving behavior. When abnormal driving behavior is identified, an abnormal behavior warning log containing the abnormality type and corresponding deduction value is generated. Based on the abnormal behavior warning log, the basic driving behavior score is deducted repeatedly to generate a comprehensive driving behavior evaluation result.

[0018] Thirdly, this application provides a computer device, including a memory and a processor. The memory stores a computer program, and the processor executes the computer program to implement any of the above-mentioned automatic evaluation methods for urban rail driver behavior based on simulated driving scenarios.

[0019] Fourthly, this application provides a computer-readable storage medium storing a computer program thereon, which, when executed by a processor, implements any of the above-mentioned automatic evaluation methods for urban rail driver behavior based on simulated driving scenarios.

[0020] In summary,

[0021] To better understand and implement this invention, the following detailed description is provided in conjunction with the accompanying drawings. Attached Figure Description

[0022] Figure 1 A flowchart illustrating an automatic evaluation method for urban rail driver behavior in a simulated driving scenario, provided in an embodiment of this application;

[0023] Figure 2 This is a schematic diagram of the structure of an automatic evaluation system for urban rail driver behavior based on a simulated driving scenario, provided as another embodiment of this application. Detailed Implementation

[0024] To facilitate understanding of the present invention, a more complete description will be given below with reference to the accompanying drawings. Preferred embodiments of the invention are shown in the drawings. However, the invention can be implemented in many different forms and is not limited to the embodiments described herein. Rather, these embodiments are provided to provide a thorough and complete understanding of the disclosure of the invention.

[0025] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this invention pertains. The terminology used herein in the description of the invention is for the purpose of describing particular embodiments only and is not intended to be limiting of the invention. The term "and / or" as used herein includes any and all combinations of one or more of the associated listed items.

[0026] In one embodiment, such as Figure 1 As shown, an automatic evaluation method for urban rail transit driver behavior based on simulated driving scenarios is provided. This embodiment illustrates the method by applying it to a terminal. It is understood that this method can also be applied to a server, or to a system including both a terminal and a server, and implemented through interaction between the terminal and the server. In this embodiment, the method includes the following steps:

[0027] S1: The operation signals, voice signals and vehicle dynamic signals of urban rail drivers under standard driving operation procedures are collected by the simulated driving assessment equipment. The operation signals, voice signals and vehicle dynamic signals are time-series encoded, voice feature extracted and trajectory recorded respectively, and then aligned with a unified time reference to generate a multimodal standard behavior template library.

[0028] Specifically, the system receives multi-source signals generated by urban rail drivers under standard driving procedures through the data acquisition interface of the simulated driving assessment equipment. This acquisition process covers operation signals triggered by the driver's control console, voice signals generated by the driver executing standardized joint control commands, and vehicle dynamic signals calculated in real time by the simulated driving platform. Operation signals include controller level change signals, button press signals, switch actuation signals, and brake handle displacement signals. Voice signals include a continuous audio stream when the driver reads joint control commands. Vehicle dynamic signals include simulated train speed, acceleration, deceleration, and brake cylinder pressure parameters. The system performs time-series encoding processing on the operation signals, converting discrete operation events into a binary state sequence with timestamps. Each operation event is digitally represented by its occurrence time and duration.

[0029] The system extracts speech features from the speech signal, obtaining a sequence of acoustic feature parameters reflecting the speech content through spectral analysis and acoustic parameter calculation. This sequence includes Mel-frequency cepstral coefficients and their differential coefficients. The system records the trajectory of vehicle dynamic signals, sampling and storing continuous change curves of vehicle operating parameters at fixed time intervals. After completing feature extraction for various signals, the system uses the master clock of the simulated driving assessment equipment as a unified time reference to timestamp the acquisition times of data from each channel. Through interpolation alignment methods, the operation event sequence, speech feature sequence, and vehicle dynamic trajectory sequence are mapped to the same time coordinate system, generating a multimodal standard behavior template library. This template library stores the standard operation timing features, standard voice control features, and corresponding standard vehicle dynamic response features for each operation step in the standard driving operation process.

[0030] S2: Collect time-series data of the examinee's operational behavior, speech features, vehicle dynamics, and video images during the assessment process at a preset sampling period, and then merge them after aligning them by timestamp to generate a multimodal driving behavior dataset.

[0031] Specifically, the system collects multi-dimensional driving behavior data of examinees during the assessment process at a preset sampling period. Operational behavior time-series data is acquired through sensor arrays of various control components on the driving console, including handle position sensors, button switch status sensors, and touchscreen operation recorders. The system reads the values ​​of each sensor at fixed sampling intervals and converts them into digital signals, while simultaneously recording the timestamp information corresponding to each sampling point. Voice feature time-series data is collected through a near-field microphone worn by the examinee. The system preprocesses the voice signals and extracts feature parameters, generating a sequence of voice feature vectors containing short-time energy, zero-crossing rate, and spectral envelope parameters. Each feature vector is timestamped according to its corresponding voice sampling time. Vehicle dynamic time-series data is generated and output in real time by the simulated driving platform, including the instantaneous speed, instantaneous acceleration, braking distance change rate, and traction output value of the simulated train at various locations on the track. The system synchronously collects vehicle dynamic parameters at the same sampling period as the operational behavior data and timestamps each data point. Video image data is acquired through multiple image acquisition devices installed in the simulated cockpit. These devices capture continuous image frame sequences of the examinee's facial and hand areas from the front and side, respectively. The system records image data at a fixed frame rate and timestamps each frame. After acquiring various types of data, the system uses a unified time reference to arrange and align the time-series data of operational behavior, voice features, vehicle dynamics, and video images in chronological order based on the timestamps of each data point. The aligned multi-source data is then fused along the time dimension, forming a comprehensive data record containing operational status, voice features, vehicle dynamics, and image frames for each time slice, generating a multimodal driving behavior dataset.

[0032] S3: Compare the time-series data of operation behavior, speech features, and vehicle dynamics in the multimodal driving behavior dataset with the multimodal standard behavior template library item by item, calculate the quantitative value of behavior deviation under each dimension, and generate a multi-dimensional deviation feature set containing operation deviation features, speech deviation features, and dynamic deviation features.

[0033] Specifically, the system compares the time-series data of operational behaviors in the multimodal driving behavior dataset with the standard operational time-series features stored in the multimodal standard behavior template library. The comparison process first locates the standard operational steps corresponding to the current assessment stage based on the timestamp, then extracts the candidate's actual operational sequence within that time window, and calculates the time-series deviation and state deviation between the actual operational sequence and the standard operational sequence. The time-series deviation represents the time offset of the operation occurrence relative to the standard time, while the state deviation represents the degree of deviation of the actual movement amplitude or gear position of the operational component from the standard requirements. The system compares the time-series data of voice features with standard voice control features. By calculating the similarity distance between the candidate's actual voice feature vector sequence and the corresponding standard voice feature vector sequence in the standard template library, the system obtains the integrity deviation and accuracy deviation of voice control. The system compares the time-series data of vehicle dynamics with standard vehicle dynamic response features. By calculating the area of ​​difference and peak deviation between the actual change curve of the vehicle's operating parameters during the candidate's driving process and the standard trajectory curve in the standard template library, the system obtains the deviation of the vehicle's dynamic response. After completing the item-by-item comparison of each dimension, the system aggregates the operational deviation, voice deviation, and dynamic deviation in chronological order to form a multi-dimensional deviation feature set containing operational deviation features, voice deviation features, and dynamic deviation features. This deviation feature set is indexed by the time axis, and records the deviation values ​​of each dimension at each assessment time point. The sign of the deviation value represents the direction of deviation, and the magnitude of the deviation value represents the degree of deviation.

[0034] S4: Calculate the operation standardization score, voice standardization score, and driving stability score respectively based on the multi-dimensional deviation feature set. Then, weight and fuse the operation standardization score, voice standardization score, and driving stability score according to preset weights to generate a basic driving behavior score.

[0035] Specifically, the system calculates the score for each dimension based on a multi-dimensional deviation feature set. The operational standardization score is calculated based on the temporal and state deviations within the operational deviation features. The system categorizes and summarizes the operational deviations at each assessment time point according to their type to generate an operational standardization score. The voice standardization score is calculated based on the completeness and accuracy deviations within the voice deviation features. The system groups the voice deviation features by control items, calculates the voice standardization score for each item within each group based on its deviation value, and then aggregates them according to item weights to generate an overall voice standardization score. The driving smoothness score is calculated based on the vehicle operating parameter deviations within the dynamic deviation features. The system calculates the smoothness sub-scores for each parameter based on speed deviation, acceleration deviation, and braking curve deviation, and then weights and aggregates them according to parameter importance to generate a driving smoothness score. After obtaining the scores for the three dimensions, the system performs a weighted fusion using preset weights. The weight values ​​are set according to the relative importance of operational standardization, voice standardization, and driving smoothness in the urban rail driver driving behavior evaluation standard. The weighted fusion process involves multiplying each dimension score by its corresponding weight and then summing the results to generate a basic driving behavior score. This basic score reflects the candidate's basic driving skills level excluding the influence of abnormal driving behavior, and serves as the benchmark for subsequent deductions for abnormal behavior.

[0036] S5: Based on video image data in the multimodal driving behavior dataset, extract the coordinates of key points on the driver's face and calculate the time series of eyelid opening and closing. Simultaneously, calculate the rate of change of speed and the rate of change of acceleration based on vehicle dynamic time series data. Compare the time series of eyelid opening and closing with a preset fatigue threshold to identify fatigued driving behavior, and compare the rate of change of speed and the rate of change of acceleration with a preset aggressive threshold to identify aggressive driving behavior. When abnormal driving behavior is identified, generate an abnormal behavior warning log containing the abnormal type and corresponding deduction value. Based on the abnormal behavior warning log, accumulate deductions are made on the basic driving behavior score to generate a comprehensive driving behavior evaluation result.

[0037] Specifically, the system performs facial key point detection based on video image data from a multimodal driving behavior dataset. The system localizes the facial region in each frame of the video image data, extracts the coordinates of key points around the eyes within the localized facial region, calculates the eyelid opening / closing degree based on the vertical distance between the key points of the upper and lower eyelids, and generates an eyelid opening / closing degree time series by arranging the continuous time series in chronological order. Simultaneously, the system calculates the rate of change of speed and the rate of change of acceleration based on vehicle dynamic time series data. The rate of change of speed is obtained by differential operation on the speed series, and the rate of change of acceleration is obtained by differential operation on the acceleration series. The system compares the eyelid opening / closing degree time series with a preset fatigue threshold. When the average value of the eyelid opening / closing degree per unit time is lower than the preset fatigue threshold and the duration exceeds a preset duration threshold, the system identifies the current driving state as fatigued driving behavior.

[0038] The system compares the rate of change of speed and the rate of change of acceleration with preset aggressive thresholds. When the absolute value of the rate of change of speed exceeds the preset aggressive threshold within a unit of time, and the absolute value of the rate of change of acceleration also exceeds the preset aggressive threshold, the system identifies the current driving state as aggressive driving behavior. Upon detecting abnormal driving behavior, the system generates an abnormal behavior warning log. This log includes an abnormality type identifier and a corresponding deduction value. The abnormality type identifier distinguishes between fatigued driving behavior and aggressive driving behavior, and the deduction value is set according to the severity level of various abnormal behaviors in the urban rail driver driving behavior evaluation standards. Based on the abnormality type and deduction value recorded in the abnormal behavior warning log, the system performs cumulative deductions on the basic driving behavior score. Each abnormal behavior record corresponds to one deduction operation, which is executed sequentially according to the time sequence of the abnormal behaviors during the assessment. The final score is output as the comprehensive driving behavior evaluation result.

[0039] In summary, the automatic evaluation method for urban rail transit driver behavior based on simulated driving scenarios provided in this application establishes a unified quantitative reference benchmark for driving operation, voice control, and vehicle dynamic response by constructing a multimodal standard behavior template library. This fundamentally eliminates the problem of inconsistent scoring scales caused by subjective differences among evaluators in traditional manual evaluation, making the evaluation results repeatable and objectively comparable. By synchronously collecting and merging four types of data—operational behavior, voice features, vehicle dynamics, and video images—with timestamps, a precise millisecond-level time correlation between driving operation and vehicle dynamic response can be achieved. This addresses the technical deficiency of difficulty in tracing deductions to specific operation moments due to the dispersion and isolation of multi-source data, ensuring complete data traceability throughout the evaluation process. By comparing real-time collected candidate data with standard templates item by item and calculating the quantified deviation values ​​under each dimension, the method can effectively address the abstract operational standardization, voice compliance, and driving behavior. Driving smoothness is transformed into a calculable mathematical indicator, realizing the transformation of driving behavior from qualitative evaluation to quantitative assessment. By detecting facial key points based on video image data and calculating the time series of eyelid opening and closing to identify fatigue driving behavior, and by calculating the rate of change of speed and acceleration based on vehicle dynamic time series data to identify aggressive driving behavior, real-time detection and automatic warning of abnormal driving behavior can be achieved during the assessment process, overcoming the limitations of traditional assessments that rely on examiners' real-time observation to identify risky behaviors. By cumulatively deducting points from the basic driving behavior score based on abnormal behavior warning logs, the impact of abnormal driving behavior on safety can be quantified as score deductions, so that the final assessment result comprehensively reflects the candidate's overall driving safety level. At the same time, the encrypted storage of the assessment record package supports complete playback according to multi-dimensional search conditions, which can provide objective operation records and scoring basis when appealing the results, thereby improving the credibility and scientific nature of skills assessment.

[0040] In one embodiment, S1 of the automatic evaluation method for urban rail transit driver behavior based on simulated driving scenarios provided by the present invention specifically includes the following steps:

[0041] S11: Divide the operation signals into time windows according to the stage labels of the assessment process, and splice and normalize the handle displacement sequence, button pressing sequence and brake level change sequence in each time window according to the operation sequence to generate the standard operation sequence template corresponding to each stage.

[0042] S12: Perform silence segment removal and endpoint detection processing on the speech signal, extract the Mel frequency cepstral coefficient feature vector sequence from the effective speech segments after removing silence segments, classify and aggregate each effective speech segment according to semantic labels, take the mean vector of each dimension, and generate the standard speech template corresponding to each semantic label.

[0043] S13: The vehicle dynamic signal is segmented according to the operating condition label. The speed curve, acceleration curve and benchmark distance curve in each segment are Gaussian smoothed and filtered to extract the common trajectory baseline and generate the standard dynamic trajectory template corresponding to each operating condition.

[0044] S14: Cross-index and associate the standard operation sequence template, standard voice template, and standard dynamic trajectory template according to the stage labels and operating condition labels of the assessment process to generate a multimodal standard behavior template library containing a three-dimensional mapping relationship between operation, voice, and trajectory.

[0045] Specifically, the system divides the operation signal sequence into multiple time windows based on predefined stage labels in the assessment process, with each time window corresponding to a standard operation stage. The system extracts handle displacement, button status, and braking level values ​​from each time window to form a multi-dimensional operation feature vector. The handle displacement value is collected by a position sensor and is expressed in angles or percentages; the button status value is a binary quantity; and the braking level value is a discrete number of levels. Because the dimensions and value ranges of each operation feature differ, the system normalizes each dimension using a linear mapping to transform the original values ​​to a unified range. The normalization factor for handle displacement is the difference between the handle's maximum and minimum physical travel; the normalization factor for braking level is the difference between the maximum and minimum number of levels; and the button status retains its binary value. Preferably, the normalization formula is:

[0046]

[0047] in, For the first The original sampled values ​​of each operational feature, and These represent the lower and upper physical limits of this feature in standard operating procedures. The normalized dimensionless value ranges from 0 to 1. The physical basis of this normalization is to ensure that operation signals with different dimensions have a consistent numerical scale in subsequent time-series splicing and comparison, avoiding bias in deviation calculations caused by differences in magnitude. The system splices the normalized handle displacement sequence, button state sequence, and brake level change sequence within the same time window according to the time sequence of operation occurrence, forming the operation time-series vector sequence for that stage. Each time point in the spliced ​​sequence corresponds to a multi-dimensional operation state vector. The system directly uses the spliced ​​sequence as the standard operation time-series template for that stage. The template length is the number of sampling points within the total duration of that stage, and each vector in the template represents the standard state of each operating component at that moment.

[0048] The system performs silence segment removal and endpoint detection on the speech signal, calculates the short-time energy and short-time zero-crossing rate of the speech signal, and uses a dual-threshold determination method to identify the start and end frames of valid speech segments. Consecutive frames with energy below the low energy threshold and a zero-crossing rate below the low zero-crossing rate threshold are identified as silence segments and removed; the remaining portion is considered a valid speech segment. For each valid speech segment, the system performs pre-emphasis, framing, and Hamming windowing. Each frame has a fixed time window length, and the frame shift is half the window length. A Fast Fourier Transform is performed on each frame, and the spectrum is passed through a Mel filter bank to obtain the Mel spectral energy. After logarithmic operations and discrete cosine transforms, the Mel frequency cepstral coefficient feature vector is obtained. The system extracts the static coefficients and their first and second differences to form the complete feature vector for each frame. For each valid speech segment, the system classifies it according to its corresponding semantic tags, which include standard control terms and command names. The system calculates the arithmetic mean of the feature vectors of each frame for all valid speech segments under the same semantic tag, obtaining the standard feature vector of that semantic tag. The formula for calculating the mean is as follows:

[0049]

[0050] in, For the first The first valid speech segment in the... MFCC feature vector of the frame, The number of frames in this segment. For belonging to the first The total number of segments with semantic tags. For the first The system stores the standard feature vector of each semantic label. The physical meaning of this mean is the central position of all pronunciation samples under the same semantic label in the acoustic feature space, reflecting the common acoustic pattern of standard pronunciation. Its dimensions are consistent with the MFCC coefficients. The system stores each semantic label and its corresponding standard feature vector as a standard speech template.

[0051] Furthermore, the system divides the vehicle's dynamic signals into multiple segments based on operating condition labels, which include categories such as traction, cruise, coasting, braking, and stopping. The speed curve within each segment is then analyzed. Acceleration curve and benchmark distance curve Gaussian smoothing filtering was applied separately to remove sensor noise and simulation fluctuations. Gaussian smoothing employed a one-dimensional Gaussian kernel function. ,in The standard deviation of the Gaussian kernel, measured in units of time, controls the smoothing bandwidth. The smoothed velocity curve is generated by convolution operation. Similarly, the acceleration and smoothed distance curves are obtained. The system aggregates all smoothed curve samples under this condition by time point, calculates the arithmetic mean of each sample at each time point, and obtains the common trajectory baseline. The formula for the mean baseline is: ,in This represents the total number of sample curves under this operating condition. This is the standard velocity trajectory, measured in meters per second. The system performs the same operation on acceleration and reference distance to obtain the standard acceleration trajectory. The unit is meters per second², and the standard benchmark distance trajectory is... The unit is meters. The physical meaning of this mean baseline is the typical variation trend of vehicle dynamic parameters in multiple operation records under the same operating conditions, eliminating random fluctuations and individual differences. The system combines the standard speed trajectory, standard acceleration trajectory, and standard benchmark distance trajectory under each operating condition to generate a standard dynamic trajectory template for that operating condition.

[0052] The system cross-indexes and associates the generated standard operation sequence templates, standard voice templates, and standard dynamic trajectory templates. The system establishes a three-dimensional mapping table using the stage labels and operating condition labels of the assessment process as joint index keys. Stage labels identify the sequence of operation steps and voice control, while operating condition labels identify the vehicle's power status. The system associates the operation sequence templates under the same stage label with the voice templates for that stage, as that stage includes both specific operational actions and corresponding control phrases. Simultaneously, the system associates the relevant dynamic trajectory templates with the aforementioned operation and voice templates based on the corresponding operating condition label for that stage. Each associated entry contains a stage label, an operating condition label, an operation sequence template pointer, a voice template pointer, and a dynamic trajectory template pointer, with pointers indicating the starting address of each template in the storage space. The system organizes all associated entries into a sequence according to the chronological order of the assessment process, forming a multimodal standard behavior template library. The storage structure of this template library is an index table, allowing the system to quickly retrieve the corresponding operation sequence template, voice feature template, and dynamic trajectory template through combined queries using stage labels and operating condition labels. In the template library, the timelines of the operation sequence template, voice template, and dynamic trajectory template are all set with the start time of the assessment as the zero point and a uniform sampling frequency as the time reference, ensuring that the time points of the three types of templates under the same index are aligned. The system uses the template library as a standard reference system for subsequent comparison of candidates' driving behavior. Each record in the template library represents a standard multimodal data set of a standardized driving behavior unit.

[0053] In one embodiment, S2 of the automatic evaluation method for urban rail transit driver behavior based on simulated driving scenarios provided by the present invention specifically includes the following steps:

[0054] S21: Based on a unified hardware clock source, mark the first timestamp for each frame of operation events in the operation behavior timing data, and mark the second timestamp for each frame of dynamic parameters in the vehicle dynamic timing data. Establish an operation-dynamic data association pair by aligning the time axis of the first timestamp and the second timestamp.

[0055] S22: Perform sliding window accumulation processing on each frame of speech feature vector in the speech feature time series data, concatenate the continuous speech feature vectors in the sliding window into speech feature sequence segments in time order, and extract the key frame image sequence according to the frame rate after timestamp labeling each frame of the video image data.

[0056] S23: Based on the time axis of the operation-dynamic data association pair, the speech feature sequence fragments and key frame image sequences are mapped to the same time grid through linear interpolation, so that the operation behavior time series data, vehicle dynamic time series data, speech feature time series data and video image data have aligned data samples at each time grid node.

[0057] S24: The aligned operation behavior time-series data, voice feature time-series data, vehicle dynamic time-series data, and video image data are vectorized and concatenated according to the order of time grid nodes to generate a multimodal driving behavior dataset indexed by the time axis.

[0058] Specifically, the system uses a unified hardware clock source as the time reference for the entire data acquisition system. This hardware clock source generates continuously increasing clock pulse signals with a fixed clock period. All data acquisition channels are triggered to sample at the rising edge of this clock source. The system timestamps each frame of operation events in the operation behavior timing data. Operation events include changes in handle position, button presses, and brake level switching. Each frame of operation events corresponds to a collection moment. The system reads the count value of the hardware clock source at that moment as the first timestamp, with the clock count value at the start of the assessment being zero. The system also timestamps each frame of dynamic parameters in the vehicle dynamic timing data. Dynamic parameters include instantaneous speed, acceleration, and brake cylinder pressure. The system reads the count value of the same hardware clock source at the parameter collection moment as the second timestamp.

[0059] The system performs alignment operations using the first and second timestamps as references. The alignment process iterates through all operation event frames and dynamic parameter frames, sorting and matching them according to their timestamp values. Operation event frames with an absolute timestamp difference less than a preset neighborhood radius are paired with dynamic parameter frames to establish operation-dynamic data association pairs. The operation events and dynamic parameters in these association pairs correspond to driving behaviors and vehicle responses at the same or adjacent physical moments. The physical basis for the consistency of their timelines is a physical causal relationship between the generation of the operation signal and the change in the vehicle's dynamic response, with negligible causal delay. The system arranges the established operation event frames and dynamic parameter frames chronologically, forming an operation-dynamic joint sequence with time as the horizontal axis. Each association pair in this sequence contains the operation state vector and the corresponding vehicle dynamic state vector at that moment.

[0060] The system performs sliding window accumulation processing on the speech feature time-series data. The speech feature time-series data consists of MFCC feature vectors extracted from each frame of speech signal, with the time interval between feature vectors equal to the frame shift duration of the speech frame. The system sets the window length and step size of the sliding window. The window length is in units of time, and the step size is smaller than the window length to maintain data overlap between sliding windows. At each sliding window position, the system concatenates the speech feature vectors of multiple consecutive frames covered by the window in chronological order into a speech feature sequence segment. The concatenated segment retains the original chronological order of each feature vector within the window. Simultaneously, the system timestamps the video image data. The video image data consists of consecutive frames captured by an image acquisition device at a fixed frame rate. At the time of acquisition, the hardware clock source count value is synchronously recorded as the image timestamp for each frame. The system extracts keyframe image sequences at a preset frame rate. The extraction operation involves extracting one frame every fixed number of frames; the extracted frame is designated as a keyframe, and the timestamp of the keyframe image is the acquisition time of the original frame. After extracting keyframes, the system discards unselected non-keyframes from the original image sequence to reduce data volume. The time interval of the keyframe sequence is the frame period corresponding to the extraction frame rate. The physical basis for sliding window processing of speech feature sequence segments is that speech control content usually spans multiple short frames. Sliding window accumulation can capture the continuous change pattern of speech features in the time dimension, providing complete speech segment units for subsequent comparison with standard templates.

[0061] The system establishes a unified time grid based on the time axis of the operation-dynamic data association pair. This time grid consists of a series of equally spaced time nodes, with the node interval set to be less than the minimum of the original sampling period of each data source, ensuring that the grid density is not lower than the temporal resolution of all data sources. The system maps speech feature sequence segments to this time grid using linear interpolation. For each time grid node, the system locates the two adjacent sampling points on the time axis of the speech feature sequence segment. Based on the relative position of the node time and the adjacent sampling point times, the feature vectors of the adjacent sampling points are linearly weighted and summed to obtain the interpolated feature vector at that grid node. The linear interpolation formula is:

[0062]

[0063] in, For the time of the grid node, and The sampling times for speech features adjacent to this node. and For the corresponding feature vector, the interpolation result The dimensions are the same as those of the MFCC coefficients. The system uses the same linear interpolation method for the keyframe image sequence, but since the image data is in matrix form, the interpolation is performed at the pixel level. That is, linear interpolation of grayscale values ​​is performed between corresponding pixel positions of two adjacent keyframes to generate interpolated image frames at the grid nodes. After mapping, the operation behavior time series data, vehicle dynamic time series data, speech feature time series data, and video image data all have aligned data samples at each time grid node, and the timestamps of each sample are consistent with the time of the grid node.

[0064] The system performs vector concatenation on the aligned data across dimensions according to the order of the time grid nodes. At each time grid node, the system extracts the operation state vector for that moment from the operation behavior time-series data. This vector contains normalized handle displacement values, button state values, and braking level values. It also extracts the dynamic parameter vector for that moment from the vehicle dynamic time-series data, containing velocity values, acceleration values, and reference distance values. Furthermore, it extracts the MFCC feature vector for that moment from the speech feature time-series data and the interpolated image frame matrix for that moment from the video image data. The system then performs horizontal concatenation on the operation state vector, dynamic parameter vector, and speech feature vector in dimensional order to generate a composite feature vector whose dimension is the sum of the dimensions of the three vectors. The video image matrix does not participate in the vector concatenation but is treated as an independent data item, forming a complete record for that grid node along with the composite vector and the timestamp. The system performs the above concatenation operation sequentially on all time grid nodes, with each record containing a timestamp, a composite feature vector, and an image matrix. The system arranges these records in chronological order, forming a multimodal driving behavior dataset with the time axis as the unique index. Each record in this dataset represents a comprehensive set of information about the examinee's operational status, vehicle dynamic response, voice features, and facial image at the corresponding moment. The time interval between adjacent records in the dataset is equal to the node spacing of the time grid. The system stores this dataset in non-volatile storage media as a data source for subsequent deviation comparison and anomaly detection.

[0065] In one embodiment, S3 of the automatic evaluation method for urban rail driver behavior based on simulated driving scenarios provided by the present invention specifically includes the following steps:

[0066] S31: Dynamically warp the handle level sequence in the operation behavior time series data and the standard handle level sequence in the multimodal standard behavior template library. By constructing the cumulative distance matrix and backtracking the optimal warping path, the time alignment mapping relationship is obtained. Based on the time alignment mapping relationship, the level deviation value and time delay deviation value at each time point are calculated to generate operation deviation characteristics.

[0067] S32: Perform dynamic time warping matching on each speech feature vector sequence segment in the speech feature time series data and each standard speech feature vector in the multimodal standard behavior template library. Calculate the minimum warping distance between each segment and each standard template. Mark segments with a minimum warping distance lower than a preset distance threshold as compliant speech segments and segments with a minimum warping distance higher than a preset distance threshold as deviated speech segments. Calculate the duration ratio of compliant speech segments and the number of deviated speech segments to generate speech deviation features.

[0068] S33: Perform point-by-point difference calculation on the speed curve, acceleration curve, and benchmark distance curve in the vehicle dynamic time series data and the corresponding standard speed curve, standard acceleration curve, and standard benchmark distance curve in the multimodal standard behavior template library to obtain the speed deviation curve, acceleration deviation curve, and benchmark distance deviation curve. Extract the peak speed deviation and root mean square speed deviation from the speed deviation curve, and extract the parking benchmark deviation from the benchmark distance deviation curve to generate dynamic deviation features;

[0069] S34: Synchronously associate the operational deviation features, speech deviation features, and dynamic deviation features according to their respective timestamps to form a multi-dimensional deviation feature set indexed by a unified time axis.

[0070] Specifically, the system performs dynamic time warping on the handle-level sequence in the operational behavior time series data and the standard handle-level sequence in the multimodal standard behavior template library. Dynamic time warping is used to solve the similarity measurement problem when the two time series have local scaling or offset on the time axis. The system constructs a cumulative distance matrix, where the row index of the matrix corresponds to the time point of the examinee's handle-level sequence, the column index corresponds to the time point of the standard handle-level sequence, and the matrix elements... Indicates the candidate sequence number The point and the standard sequence The sum of the local distance and the cumulative distance between points. The local distance is calculated using Euclidean distance. ,in For candidates in the first The normalized handle level bit value at time, dimensionless. For the standard template in the first The normalized handle level value at each time step is dimensionless. The cumulative distance matrix is ​​derived recursively using dynamic programming. Calculation, initial conditions The system starts from the bottom right corner of the matrix. Start backtracking, selecting the forward direction with the smallest cumulative distance at each step, until the top left corner is reached. The backtracking path is the optimal regular path, which gives the corresponding mapping relationship between the candidate sequence and the standard sequence at each time point.

[0071] Based on this mapping relationship, the system calculates the grade value of each mapping pair for the examinee. Compared with standard level value The difference is taken as the level deviation value at that time point. This difference is dimensionless and characterizes the degree of deviation in the handle position. The system simultaneously calculates the time index for each mapping pair of examinees. With standard time index The time difference after conversion of the sampling period is used as the timing delay deviation value. ,in The sampling period is in seconds. The positive or negative value indicates whether the candidate's operation is lagging behind or ahead of the standard time. The system arranges the level deviation value and timing delay deviation value of all time points in chronological order to generate operation deviation characteristics.

[0072] The system performs dynamic time warping matching on each speech feature vector sequence segment from the speech feature time-series data and each standard speech feature vector from the multimodal standard behavior template library. Each speech feature vector sequence segment consists of multiple consecutive frames of MFCC feature vectors, with a segment length of [missing information]. Frame, each frame's feature vector dimension is The length of the feature vector sequence of the standard template is Frames with the same dimension. Preferably, the system constructs a cumulative distance matrix. Local distances can be represented by cosine distances:

[0073]

[0074] in, For the candidate's passage number The feature vector of the frame, For standard template No. The frame's feature vector, with dimensionless cosine distance ranging from 0 to 2, represents the degree of difference between two vectors in direction. The system follows a recursive formula. Calculate the cumulative distance, and the final normalized distance is: The system calculates the normalized distance between the segment and each standard template corresponding to all semantic tags, and selects the minimum normalized distance as the matching distance for that segment. The system considers segments with a minimum normalized distance lower than a preset distance threshold. Segments exceeding a certain threshold are marked as compliant audio segments, while those exceeding the threshold are marked as deviant audio segments. The duration percentage of compliant audio segments is calculated as the total duration of all compliant segments. Divide by the total assessment time ,Right now This ratio is dimensionless. The number of deviation speech segments. The system directly counts the number of all segments marked as deviations. and As a speech deviation feature output, this feature characterizes the overall deviation of speech control in terms of content completeness and pronunciation accuracy.

[0075] The system calculates the point-by-point difference between the velocity curve, acceleration curve, and benchmark distance curve in the vehicle dynamic time-series data and the corresponding standard curves in the multimodal standard behavior template library. (Velocity curve) Compared with the standard speed curve At each point in time The difference The unit is meters per second. Acceleration curve difference. The unit is meters per second². (Difference between the benchmark and the curve) The unit is meters. The system uses the above difference sequence as the velocity deviation curve, acceleration deviation curve, and benchmark distance deviation curve, respectively. The system extracts the peak velocity deviation from the velocity deviation curve. The unit is meters per second, representing the maximum deviation of speed during the entire assessment period; the root mean square speed deviation is extracted. The unit is meters per second, representing the overall average magnitude of speed deviation. The system extracts the parking alignment deviation from the alignment distance deviation curve. The unit is meters, where At the moment of stopping, this value represents the absolute distance of the stopping position from the standard stopping point. The system uses peak speed deviation, root mean square speed deviation, and stopping alignment deviation as dynamic deviation features. The dimensions of these deviation features are consistent with the corresponding physical quantities, and their values ​​directly reflect the degree of deviation between the vehicle's trajectory and the standard trajectory, providing a quantitative basis for subsequent stability scoring.

[0076] The system synchronously associates operational deviation features, voice deviation features, and dynamic deviation features according to their respective timestamps. The level deviation value within the operational deviation features... and timing delay deviation value All timestamps are based on the moment the operation event occurred. Compliance percentage of voice deviation features. and deviation quantity While it is a global statistic, each compliant or deviation segment records its start and end timestamps. The system marks all time points within the segment's time interval with their corresponding status. The velocity deviation curve is part of the dynamic deviation characteristics. Acceleration deviation curve Deviation curve of the reference distance In continuous time The system uses a unified timeline from the multimodal driving behavior dataset as its index. This timeline contains all equally spaced time nodes from the start to the end of the assessment.

[0077] For each time point, the system checks whether there is an operational deviation value at that moment. If so, it assigns an operational deviation attribute to that point. It also checks whether the moment falls within the time interval of a certain voice deviation segment; if so, it assigns a corresponding voice deviation marker. Simultaneously, it extracts the speed deviation, acceleration deviation, and alignment distance deviation values ​​for that moment from the dynamic deviation curve. The system combines the operational deviation, voice deviation status, and dynamic deviation value at each time point into a deviation record, with the timestamp of that node representing the time. The system arranges all deviation records at all nodes in chronological order, forming a multi-dimensional deviation feature set indexed by a unified time axis. This feature set contains operational deviation, voice deviation status, and dynamic deviation information at each time point, comprehensively reflecting the deviation distribution of the candidate's multi-dimensional driving behavior throughout the entire assessment process.

[0078] In one embodiment, S4 of the automatic evaluation method for urban rail driver behavior based on simulated driving scenarios provided by the present invention specifically includes the following steps:

[0079] S41: Based on the level deviation value and time delay deviation value in the operational deviation characteristics, the level deviation values ​​of each level are integrated and accumulated throughout the entire assessment period to obtain the total level deviation integral, and the absolute value of each time delay deviation value is accumulated to obtain the total time delay cumulative amount. The total level deviation integral and the total time delay cumulative amount are mapped to the percentage interval respectively and then weighted and summed to generate the operational standardization score.

[0080] S42: Based on the proportion of compliant speech segment duration and the number of deviated speech segments in the speech deviation features, the proportion of compliant speech segment duration is mapped to a percentage to obtain a speech coverage score, and the number of deviated speech segments is converted to obtain a speech deviation deduction score. The speech deviation deduction score is deducted from the speech coverage score to generate a speech standardization score.

[0081] S43: Based on the speed deviation curve, acceleration deviation curve and parking benchmark deviation value in the dynamic deviation characteristics, the root mean square value of the speed deviation curve and acceleration deviation curve is calculated and then weighted and summed to obtain the stability deviation index. The stability deviation index is mapped to the percentage range through the exponential decay function to obtain the stability base score. The benchmark deviation deduction component proportional to the parking benchmark deviation value is deducted from the stability base score to generate the driving stability score.

[0082] S44: Based on the preset dimension weight allocation table corresponding to the candidate's application level, extract the weights for operational standardization, voice standardization, and driving stability, respectively. Multiply the operational standardization score, voice standardization score, and driving stability score by their respective weights and then sum them up to generate a basic driving behavior score.

[0083] Specifically, the system quantifies the level deviation and timing delay deviation values ​​in the operational deviation characteristics. The level deviation value is defined as the difference between the candidate's actual operating level and the standard required level; this difference is dimensionless and characterizes the degree of deviation of the driver's controller handle position. The timing delay deviation value is defined as the time difference between the candidate's operation and the standard operation time, in seconds, characterizing the degree of lead or lag in the operation response. The system integrates and accumulates all level deviation values ​​throughout the entire assessment period; the mathematical expression for integration and accumulation is as follows: ,in For a moment The level deviation value, The total time spent is used to assess the points. The dimension of is seconds, and its physical meaning is the cumulative effect of level deviation over time, reflecting the total amount of continuous deviation caused by insufficient operational precision. The system performs absolute value accumulation processing on each timing delay deviation value, and the accumulation expression is: ,in For the first Timing delay deviation value of the next operation The total number of operations performed within the assessment period is calculated by summing the results. The dimension of is seconds, and its physical meaning is the absolute total of all operational response delays, reflecting the overall degree of deviation of the driver in time control.

[0084] Furthermore, the system integrates the total level deviation. Mapping to a percentage range is based on a pre-set upper limit for the integral of grade deviation. and lower limit value The mapping function is The physical basis of this mapping is that the smaller the integral of the deviation, the closer the operation is to the standard; after mapping, the score decreases linearly as the deviation increases. The system accumulates the total timing delay. Mapped to a percentage range, the mapping is based on the upper limit of cumulative delay. and lower limit The mapping function is The system will map the level scores. and time series score Perform a weighted summation, with weighting coefficients and Based on the presupposition of the relative importance of grade deviation and timing delay in the operational standardization assessment, and satisfying the following: Generate operational standardization scores The score is dimensionless and ranges from 0 to 100. The higher the value, the better the standardization of the operation.

[0085] Preferably, the system scores the proportion of compliant speech segments and the number of compliant speech segments in the speech deviation features. A compliant speech segment is defined as a continuous speech segment in which the similarity distance between the examinee's actual speech features and the standard speech features is less than a preset threshold, and the total duration of this segment is [not specified]. Divide by the total assessment time Obtaining compliance percentage This ratio is dimensionless, ranging from 0 to 1, and its physical meaning is the proportion of time during the assessment where voice-activated communication met the standards. The system maps the compliance percentage to a percentage, and the mapping function is: The physical basis for this mapping is that the compliance rate is directly proportional to the voice coverage score, with full compliance earning a perfect score. The system also considers the number of deviation voice segments. A deduction conversion process is performed, defining a deviation speech segment as a continuous speech segment whose similarity distance is higher than a preset threshold and whose duration exceeds the shortest deviation duration. The deduction conversion function is as follows: ,in This is the deduction coefficient for each deviation in the speech segment, measured in points per instance, and determined by the severity level of the speech deviation in the evaluation criteria. The system uses the speech coverage score... Subtracting speech deviation components Generate speech normativity score The score is dimensionless, ranging from 0 to 100. When the score is less than 0, the system sets it to 0 to maintain the lower limit of the score. The physical meaning of this score is to penalize deviation segments based on the requirement of sufficient voice control coverage. The more deviations there are, the more points are deducted. The final score reflects the overall standardization level of voice control.

[0086] The system calculates the driving stability score based on the speed deviation curve, acceleration deviation curve, and parking alignment deviation value from the dynamic deviation characteristics. The speed deviation curve is defined as the examinee's real-time speed... Compared with the standard speed curve difference The unit is meters per second. The acceleration deviation curve is defined as the examinee's real-time acceleration. Compared with the standard acceleration curve difference The unit is meters per second². The parking alignment deviation value is defined as the remaining distance to the target at the moment the examinee stops. Standard parking distance difference The unit is meters. The root mean square value of the system's calculated velocity deviation. Its dimension is meters per second, and its physical meaning is the average amplitude of the velocity deviation over the entire assessment period; the root mean square value of the acceleration deviation is calculated. Its dimension is meters per second², and its physical meaning is the average magnitude of the acceleration deviation. The system will... and By performing a weighted summation, the stationarity deviation index is obtained. The weighting coefficient , , The weights for velocity deviation, acceleration deviation, and calibration deviation are respectively assigned, with the dimensional normalization coefficient implicitly included in the weights. This ensures that all units are consistent before summing. The system will use the stationarity deviation index Mapping to a percentage range using an exponential decay function, the mapping function is: ,in The decay coefficient, whose dimension is the reciprocal of the index unit, is based on the physical principle that a smaller index indicates better stability. Exponential decay ensures that the score monotonically decreases as the deviation increases and remains positive throughout. The system starts from the basic stability score. The calibration deviation deduction is proportional to the absolute value of the calibration deviation during parking. ,in To align with the deduction coefficients, a driving stability score is generated, measured in points per meter. The score is dimensionless, ranging from 0 to 100, and reflects the overall level of vehicle smoothness and parking accuracy.

[0087] The system extracts the corresponding operational standardization weights from a preset dimension weighting table based on the driver's skill level declared by the candidate. Speech normativity weight And driving stability weight The weighting table is set according to the different requirements for each dimension of ability in the urban rail transit driver professional skill level standard. The sum of the weights for each level satisfies the following conditions. The system will score the compliance of operations. Speech standardization score and driving stability score Each component is multiplied by its corresponding weight and then summed to generate a basic driving behavior score. The basic score is a dimensionless percentage score, and its physical meaning is the weighted comprehensive performance of the examinee's various abilities under normal driving conditions according to the level requirements. In one embodiment, the system uses a formula to calculate the stability deviation index, which is uniformly expressed in integral form as follows:

[0088]

[0089] in, It is a stationarity deviation index, and its dimensions are dimensionless after weight normalization; The total assessment time is measured in seconds. The real-time speed curve of the examinee, in meters per second; This is a standard velocity curve, with units in meters per second; The real-time acceleration curve of the examinee is shown in meters per second². This is a standard acceleration curve, with units of meters per second². The remaining distance for the examinee at the moment of parking, in meters; Standard parking alignment distance, in meters; , , These are the velocity deviation weighting coefficient, acceleration deviation weighting coefficient, and calibration deviation weighting coefficient, respectively. All three are dimensionless and satisfy the following conditions: In this formula, the root mean square of speed deviation represents the overall magnitude of speed tracking error, the root mean square of acceleration deviation represents the overall magnitude of jerk changes, and the absolute value of the calibration deviation represents the absolute error in parking position accuracy. The weighted synthesis of these three factors is based on the fact that stability assessment must simultaneously consider the stability of operating speed, the smoothness of acceleration changes, and the accuracy of parking calibration. The weighting coefficients reflect the relative proportions of their contributions to stability. This index is used for subsequent exponential decay mapping; the smaller the value, the better the driving stability, and the higher the mapped score.

[0090] In one embodiment, S5 of the automatic evaluation method for urban rail driver behavior based on simulated driving scenarios provided by the present invention specifically includes the following steps:

[0091] S51: Extract the coordinates of multiple key points in the eye region from video image data, calculate the ratio of the Euclidean distance between the key points of the upper and lower eyelids to the Euclidean distance between the key points of the inner and outer corners of the eyes in each frame image, generate the eyelid opening and closing time series, perform sliding window mean filtering on the eyelid opening and closing time series and compare it with the preset fatigue threshold point by point, and mark the time periods that are continuously lower than the preset fatigue threshold as candidate fatigue driving time periods;

[0092] S52: Extract the speed difference and acceleration difference between adjacent time points from the vehicle dynamic time series data. Divide the speed difference by the time interval to obtain the speed change rate sequence and the acceleration difference by the time interval to obtain the acceleration change rate sequence. Calculate the absolute value of the speed change rate sequence and the acceleration change rate sequence respectively and compare them point by point with the preset aggressive threshold. Mark the time period when the absolute value of the speed change rate or the absolute value of the acceleration change rate exceeds the preset aggressive threshold as an aggressive driving candidate time period.

[0093] S53: Perform duration accumulation processing on candidate fatigue driving periods. When the duration of a single candidate fatigue driving period exceeds a preset first duration threshold, it is confirmed as a fatigue driving event. Perform duration accumulation processing on candidate aggressive driving periods. When the duration of a single candidate aggressive driving period exceeds a preset second duration threshold, it is confirmed as an aggressive driving event.

[0094] S54: Based on the preset mapping table between abnormal types and deduction values, query the first deduction value corresponding to the fatigue driving event and the second deduction value corresponding to the aggressive driving event, summarize each abnormal event and its corresponding deduction value to generate an abnormal behavior warning log, deduct the deduction values ​​corresponding to each abnormal event in the abnormal behavior warning log from the basic driving behavior score in sequence, and generate a comprehensive driving behavior evaluation result.

[0095] Specifically, the system extracts key point coordinates of the driver's facial region from video image data in a multimodal driving behavior dataset. The system employs a convolutional neural network-based facial key point detection model for inference on each frame. This model contains five convolutional layers and two fully connected layers. The convolutional layers use 3×3 kernels and 2×2 max pooling, with the number of kernels in each layer being 32, 64, 128, 256, and 512, respectively. The number of nodes in the fully connected layers is 1024 and 136, respectively. The output is the horizontal and vertical coordinate values ​​of 68 facial key points. During model training, a facial image dataset labeled with 68 key points is used. The loss function is the mean squared error between the predicted and ground truth coordinates. The optimizer uses adaptive moment estimation, with the initial learning rate and batch size set to preset values. Training is iterated until the loss function converges on the validation set. The system extracts multiple key points from the detection results for the left and right eye regions, including the upper and lower eyelid boundaries and the inner and outer corners of the eyes. For each frame of the image, the system calculates the Euclidean distance between the key points of the upper eyelid and the corresponding key points of the lower eyelid, and takes the average of the distances between all corresponding point pairs as the distance between the upper and lower eyelids. The unit is pixels; at the same time, the Euclidean distance between the inner and outer corners of the eye is calculated as the inner and outer corner distance. The unit is pixels.

[0096] The system calculates the ratio of the distance between the upper and lower eyelids to the distance between the inner and outer corners of the eyes. This ratio is dimensionless and its physical meaning is a normalized measure of eyelid opening and closing relative to eye size, used to eliminate the influence of individual differences and changes in head distance on absolute distance. The system arranges the ratios of all frames in chronological order to generate a time series of eyelid opening and closing. The system performs sliding window mean filtering on the sequence, with the sliding window length set to a preset number of frames. The filtered sequence is as follows: ,in The number of frames within the window. The frame interval is in seconds. The dimensions of the filtered sequence remain unchanged, still a dimensionless ratio. The system compares the filtered eyelid opening / closing degree with a preset fatigue threshold. Perform point-by-point comparison, when The time interval marked as a candidate point for fatigue state is recorded as a candidate period for fatigue driving.

[0097] The system extracts velocity and acceleration values ​​at adjacent time points from vehicle dynamic time-series data. The system uses a fixed sampling period. Acquisition velocity sequence and acceleration sequence ,in The sampling point number, The unit is seconds. The system calculates the velocity change rate sequence. The dimension of this rate of change is meters per second², and its physical meaning is the change in velocity of the vehicle per unit time, i.e., the actual acceleration. The system calculates the acceleration rate of change sequence. The dimension of this rate of change is meters per second³, and its physical meaning is the change in vehicle acceleration per unit time, i.e., jerk, which characterizes the impact of driving operations. The system obtains the absolute value of the velocity rate of change sequence. Taking the absolute value of the acceleration rate of change sequence yields The system will Compared with the preset aggressive threshold For comparison, the threshold's dimension is meters per second², while... Compared with the preset aggressive threshold For comparison, the unit of this threshold is meters per second³. When or When the sampling point is marked as an aggressive state candidate point, the system records the time intervals corresponding to consecutive sampling points marked as aggressive state candidate points as aggressive driving candidate time periods. Exceeding the speed change rate limit indicates that the driver applies excessive traction or braking commands within a short period, while exceeding the acceleration change rate limit indicates that the driver adjusts the control level too frequently or drastically. Exceeding either limit is considered potential aggressive driving behavior, based on the fact that aggressive driving behavior typically manifests as sudden speed changes or unstable control.

[0098] The system performs duration accumulation processing on candidate fatigue driving periods. Each candidate fatigue driving period is calculated from its start time. and end time Defined, duration is The unit is seconds. The system will compare this duration with a preset first duration threshold. When comparing, When this threshold is reached, the system classifies the entire candidate period as a single instance of fatigued driving. The physical basis for this threshold setting is that brief eyelid closure may be considered normal blinking, while only sustained closure exceeding a certain duration reflects abnormal driver fatigue. The system applies the same duration accumulation processing to all aggressive driving candidate periods, with each aggressive driving candidate period having a duration of [duration value missing]. The unit is seconds. The system compares this duration with a preset second duration threshold. When comparing, When an event occurs, the system identifies the entire candidate time period as an aggressive driving event. The physical basis for setting this threshold is that instantaneous speed or acceleration fluctuations may originate from road disturbances or sensor noise; only sustained exceedances indicate a systematic deviation in the driver's intentions. Upon event confirmation, the system records the start and end times and event type for each event, categorized into fatigue driving events and aggressive driving events. The system stores all confirmed abnormal events in a temporary event list in chronological order of occurrence, with each event entry containing an event type identifier, start time, and end time.

[0099] The system queries the first deduction value corresponding to the fatigue driving event based on a preset mapping table of anomaly types and deduction values. The second penalty point corresponding to aggressive driving incidents This mapping table is set according to the severity of various abnormal behaviors in the urban rail driver driving behavior assessment standards, with deduction values ​​being dimensionless percentage deduction units. The system iterates through each abnormal event in the temporary event list. For fatigue driving events, the system obtains its corresponding first deduction value; for aggressive driving events, the system obtains its corresponding second deduction value. The system summarizes the type, occurrence time, and corresponding deduction value of each abnormal event to generate an abnormal behavior warning log. Each record in the log includes an event type field, a start time field, an end time field, and a deduction field. The system starts from the basic driving behavior score. Starting from this point, the point deduction operations will be performed sequentially according to the time order of events in the abnormal behavior warning log. The point deduction operations are as follows: ,in This represents the total number of abnormal events. For the first The system deducts points for each abnormal event. After each deduction, the system checks if the current score is lower than a preset minimum score. If it is, the minimum score is used as the final score to prevent negative scores. The physical basis for the point deduction is that abnormal driving behavior poses an additional risk to driving safety, requiring a punitive deduction from the basic skill score. The result is a comprehensive assessment of driving behavior, and its dimension is a dimensionless percentage score, which is directly output as the candidate's final score for this assessment.

[0100] It should be understood that although the steps in the flowcharts of the embodiments described above are shown sequentially according to the arrows, these steps are not necessarily executed in the order indicated by the arrows. Unless explicitly stated herein, there is no strict order restriction on the execution of these steps, and they can be executed in other orders. Moreover, at least some steps in the flowcharts of the embodiments described above may include multiple steps or multiple stages. These steps or stages are not necessarily completed at the same time, but can be executed at different times. The execution order of these steps or stages is not necessarily sequential, but can be performed alternately or in turn with other steps or at least some of the steps or stages of other steps.

[0101] Based on the same inventive concept, this application also provides an automatic evaluation system for urban rail transit driver behavior in a simulated driving scenario, used to implement the aforementioned automatic evaluation method for urban rail transit driver behavior in a simulated driving scenario. The solution provided by this system is similar to the implementation described in the above method. Therefore, the specific limitations of one or more embodiments of the automatic evaluation system for urban rail transit driver behavior in a simulated driving scenario provided below can be found in the limitations of the automatic evaluation method for urban rail transit driver behavior in a simulated driving scenario described above, and will not be repeated here.

[0102] Preferably, such as Figure 2 As shown, the present invention provides an automatic evaluation system 600 for urban rail transit driver behavior based on simulated driving scenarios. The system is configured with the following modules:

[0103] The standard behavior template construction module 610 is used to collect the operation signals, voice signals and vehicle dynamic signals of urban rail drivers under the standard driving operation process through the simulated driving assessment equipment. After performing time-series encoding, voice feature extraction and trajectory recording on the operation signals, voice signals and vehicle dynamic signals respectively, they are aligned with a unified time reference to generate a multimodal standard behavior template library.

[0104] The candidate multimodal data fusion module 620 is used to collect time-series data of the candidate's operational behavior, speech features, vehicle dynamics and video images during the assessment process at a preset sampling period, and then fuse them after aligning them by timestamp to generate a multimodal driving behavior dataset.

[0105] The multi-dimensional behavior deviation calculation module 630 is used to compare the operation behavior time-series data, voice feature time-series data, and vehicle dynamic time-series data in the multimodal driving behavior dataset with the multimodal standard behavior template library item by item, calculate the behavior deviation quantification value under each dimension, and generate a multi-dimensional deviation feature set containing operation deviation features, voice deviation features, and dynamic deviation features.

[0106] The driving basic score fusion module 640 is used to calculate the operation standardization score, voice standardization score and driving stability score according to the multi-dimensional deviation feature set, and then weight and fuse the operation standardization score, voice standardization score and driving stability score according to preset weights to generate a driving behavior basic score.

[0107] The anomaly identification and comprehensive evaluation module 650 is used to extract the coordinates of key points on the driver's face based on video image data in a multimodal driving behavior dataset and calculate the time series of eyelid opening and closing. At the same time, it calculates the rate of change of speed and the rate of change of acceleration based on vehicle dynamic time series data. It compares the time series of eyelid opening and closing with a preset fatigue threshold to identify fatigue driving behavior, and compares the rate of change of speed and the rate of change of acceleration with a preset aggressive threshold to identify aggressive driving behavior. When abnormal driving behavior is identified, it generates an abnormal behavior warning log containing the abnormality type and the corresponding deduction value. Based on the abnormal behavior warning log, it accumulates deductions on the basic driving behavior score and generates a comprehensive driving behavior evaluation result.

[0108] In one embodiment, this application also provides a computer device, including a memory and a processor, wherein the memory stores a computer program, and the processor executes the computer program to implement the above-described automatic evaluation method for urban rail driver behavior based on simulated driving scenarios.

[0109] In one embodiment, this application also provides a computer-readable storage medium storing a computer program thereon, which, when executed by a processor, implements the above-described automatic evaluation method for urban rail driver behavior based on a simulated driving scenario.

[0110] In the description of this specification, references to terms such as "one embodiment," "some embodiments," "example," "specific example," or "some examples," etc., indicate that a specific feature, structure, material, or characteristic described in connection with that embodiment or example is included in at least one embodiment or example of this application. Furthermore, the specific features, structures, materials, or characteristics described may be combined in any suitable manner in one or more embodiments or examples. Moreover, without contradiction, those skilled in the art can combine and integrate the different embodiments or examples described in this specification, as well as the features of those different embodiments or examples.

[0111] For the device embodiments, since they basically correspond to the method embodiments, the relevant parts can be referred to in the description of the method embodiments. The device embodiments described above are merely illustrative. The components described as separate parts may or may not be physically separate, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed across multiple network units. Some or all of the modules can be selected to achieve the purpose of this disclosure according to actual needs. Those skilled in the art can understand and implement this without creative effort.

[0112] The above description is merely a specific embodiment of this application, but the scope of protection of this application is not limited thereto. Any person skilled in the art can easily conceive of various variations or substitutions within the technical scope disclosed in this application, and these should all be included within the scope of protection of this application. Therefore, the scope of protection of this application should be determined by the scope of the claims.

Claims

1. A method for automatically evaluating the driving behavior of urban rail transit drivers based on simulated driving scenarios, characterized in that, Includes the following steps: S1: Collect the operation signals, voice signals and vehicle dynamic signals of urban rail drivers under the standard driving operation procedure through the simulated driving assessment equipment. After performing time-series coding, voice feature extraction and trajectory recording on the operation signals, voice signals and vehicle dynamic signals respectively, align them with a unified time reference to generate a multimodal standard behavior template library. S2: Collect time-series data of the examinee's operational behavior, speech features, vehicle dynamics, and video images during the assessment process at a preset sampling period, and then merge them after aligning them by timestamp to generate a multimodal driving behavior dataset; S3: Compare the operation behavior time-series data, voice feature time-series data, and vehicle dynamic time-series data in the multimodal driving behavior dataset with the multimodal standard behavior template library item by item, calculate the behavior deviation quantification value under each dimension, and generate a multi-dimensional deviation feature set containing operation deviation features, voice deviation features, and dynamic deviation features. S4: Calculate the operation standardization score, voice standardization score and driving stability score according to the multi-dimensional deviation feature set, and then weight and fuse the operation standardization score, the voice standardization score and the driving stability score according to the preset weight to generate a basic driving behavior score; S5: Based on the video image data in the multimodal driving behavior dataset, extract the coordinates of the driver's facial key points and calculate the eyelid opening and closing time series. Simultaneously, calculate the speed change rate and acceleration change rate based on the vehicle dynamic time series data. Compare the eyelid opening and closing time series with a preset fatigue threshold to identify fatigue driving behavior, and compare the speed change rate and acceleration change rate with a preset aggressive threshold to identify aggressive driving behavior. When abnormal driving behavior is identified, generate an abnormal behavior warning log containing the abnormal type and corresponding deduction value. Based on the abnormal behavior warning log, accumulate deductions are made on the basic driving behavior score to generate a comprehensive driving behavior evaluation result.

2. The method according to claim 1, characterized in that, S1 includes: S11: Divide the operation signal into time windows according to the stage labels of the assessment process, and splice and normalize the handle displacement sequence, button pressing sequence and braking level change sequence in each time window according to the operation sequence to generate the standard operation sequence template corresponding to each stage. S12: Perform silence segment removal and endpoint detection processing on the speech signal, extract the Mel frequency cepstral coefficient feature vector sequence from the effective speech segments after removing silence segments, classify and aggregate each effective speech segment according to semantic tags, take the mean vector of each dimension, and generate the standard speech template corresponding to each semantic tag. S13: The vehicle dynamic signal is segmented according to the operating condition label. The speed curve, acceleration curve and benchmark distance curve in each segment are Gaussian smoothed and filtered to extract the common trajectory baseline and generate the standard dynamic trajectory template corresponding to each operating condition. S14: Cross-index and associate the standard operation sequence template, the standard voice template, and the standard dynamic trajectory template according to the stage labels of the assessment process and the operating condition labels to generate a multimodal standard behavior template library containing a three-dimensional mapping relationship between operation, voice, and trajectory.

3. The method according to claim 1, characterized in that, S2 includes: S21: Using a unified hardware clock source as a reference, mark each frame of operation events in the operation behavior timing data with a first timestamp, and mark each frame of dynamic parameters in the vehicle dynamic timing data with a second timestamp. Establish an operation-dynamic data association pair by aligning the time axis of the first timestamp and the second timestamp. S22: Perform sliding window accumulation processing on each frame of speech feature vector in the speech feature time series data, concatenate the continuous speech feature vectors in the sliding window into speech feature sequence segments in time order, and extract key frame image sequences according to frame rate after timestamp labeling each frame of the video image data. S23: Using the time axis of the operation-dynamic data association pair as a reference, the speech feature sequence fragments and the keyframe image sequence are mapped to the same time grid through linear interpolation, so that the operation behavior time series data, the vehicle dynamic time series data, the speech feature time series data and the video image data have aligned data samples at each time grid node; S24: The aligned operation behavior time-series data, the voice feature time-series data, the vehicle dynamic time-series data, and the video image data are vector-concatenated according to the order of time grid nodes to generate a multimodal driving behavior dataset indexed by the time axis.

4. The method according to claim 1, characterized in that, S3 includes: S31: Perform dynamic time warping on the handle level sequence in the operation behavior time sequence data and the standard handle level sequence in the multimodal standard behavior template library. Obtain the time alignment mapping relationship by constructing a cumulative distance matrix and backtracking the optimal warping path. Calculate the level deviation value and time delay deviation value at each time point based on the time alignment mapping relationship to generate operation deviation features. S32: Perform dynamic time warping matching on each speech feature vector sequence segment in the speech feature time series data and each standard speech feature vector in the multimodal standard behavior template library, calculate the minimum warping distance between each segment and each standard template, and mark segments with a minimum warping distance lower than a preset distance threshold as compliant speech segments and segments with a minimum warping distance higher than the preset distance threshold as deviated speech segments, count the duration ratio of compliant speech segments and the number of deviated speech segments, and generate speech deviation features; S33: The speed curve, acceleration curve, and benchmark distance curve in the vehicle dynamic time series data are respectively compared with the corresponding standard speed curve, standard acceleration curve, and standard benchmark distance curve in the multimodal standard behavior template library. Point-by-point difference calculation is performed to obtain the speed deviation curve, acceleration deviation curve, and benchmark distance deviation curve. The peak speed deviation and root mean square speed deviation are extracted from the speed deviation curve, and the parking benchmark deviation is extracted from the benchmark distance deviation curve to generate dynamic deviation features. S34: Synchronously associate the operation deviation feature, the voice deviation feature, and the dynamic deviation feature according to their respective timestamps to form a multi-dimensional deviation feature set indexed by a unified time axis.

5. The method according to claim 1, characterized in that, S4 includes: S41: Based on the level deviation value and time delay deviation value in the operation deviation characteristics, the level deviation values ​​of each level are integrated and accumulated throughout the entire assessment period to obtain the total level deviation integral, and the absolute value of each time delay deviation value is accumulated to obtain the total time delay cumulative amount. The total level deviation integral and the total time delay cumulative amount are mapped to the percentage interval respectively and then weighted and summed to generate the operation standardization score. S42: Based on the proportion of compliant speech segment duration and the number of deviated speech segments in the speech deviation features, the proportion of compliant speech segment duration is mapped to a percentage to obtain a speech coverage score, the number of deviated speech segments is converted to a deduction score to obtain a speech deviation deduction score, and the speech deviation deduction score is deducted from the speech coverage score to generate a speech standardization score. S43: Based on the speed deviation curve, acceleration deviation curve, and parking benchmark deviation value in the dynamic deviation characteristics, calculate the root mean square value of the speed deviation curve and the acceleration deviation curve respectively, and then sum them by weight to obtain the stability deviation index. Map the stability deviation index to a percentage range through an exponential decay function to obtain the stability base score. Subtract the benchmark deviation deduction component that is proportional to the parking benchmark deviation value from the stability base score to generate a driving stability score. S44: Based on the preset dimension weight allocation table corresponding to the candidate's application level, extract the weights for operational standardization, voice standardization, and driving stability, respectively. Multiply the operational standardization score, the voice standardization score, and the driving stability score by their respective weights and then sum them up to generate a basic driving behavior score.

6. The method according to claim 5, characterized in that, The formula for calculating the stationarity deviation index is as follows: in, The stationarity deviation index is T, where T is the total assessment time. For the real-time speed curve of the test taker, For the standard speed curve, For the real-time acceleration curve of the examinee, For standard acceleration curves, This refers to the remaining distance for the examinee to reference when parking. This is the remaining distance relative to the standard parking distance. , , These are the weighting coefficients for velocity deviation, acceleration deviation, and calibration deviation, respectively. .

7. The method according to any one of claims 1-6, characterized in that, S5 includes: S51: Extract the coordinates of multiple key points in the eye region from the video image data, calculate the ratio of the Euclidean distance between the key points of the upper and lower eyelids to the Euclidean distance between the key points of the inner and outer corners of the eyes in each frame image, generate an eyelid opening and closing time series, perform sliding window mean filtering on the eyelid opening and closing time series and compare it point by point with a preset fatigue threshold, and mark the time periods that are continuously lower than the preset fatigue threshold as candidate fatigue driving time periods; S52: Extract the speed difference and acceleration difference between adjacent time points from the vehicle dynamic time series data, divide the speed difference by the time interval to obtain the speed change rate sequence, divide the acceleration difference by the time interval to obtain the acceleration change rate sequence, calculate the absolute value of the speed change rate sequence and the acceleration change rate sequence respectively, and compare them point by point with a preset aggressive threshold. Mark the time period when the absolute value of the speed change rate or the absolute value of the acceleration change rate exceeds the preset aggressive threshold as an aggressive driving candidate time period. S53: Perform duration accumulation processing on the candidate fatigue driving periods. When the duration of a single candidate fatigue driving period exceeds a preset first duration threshold, it is confirmed as a fatigue driving event. Perform duration accumulation processing on the candidate aggressive driving periods. When the duration of a single candidate aggressive driving period exceeds a preset second duration threshold, it is confirmed as an aggressive driving event. S54: Based on the preset mapping table between abnormal types and deduction values, query the first deduction value corresponding to the fatigue driving event and the second deduction value corresponding to the aggressive driving event, summarize each abnormal event and its corresponding deduction value to generate an abnormal behavior warning log, and deduct the deduction values ​​corresponding to each abnormal event in the abnormal behavior warning log from the basic driving behavior score in sequence to generate a comprehensive driving behavior evaluation result.

8. An automatic evaluation system for urban rail transit driver behavior based on simulated driving scenarios, characterized in that, The system includes: The standard behavior template construction module is used to collect the operation signals, voice signals and vehicle dynamic signals of urban rail drivers under the standard driving operation process through the simulated driving assessment equipment. After performing time-series encoding, voice feature extraction and trajectory recording on the operation signals, voice signals and vehicle dynamic signals respectively, they are aligned with a unified time reference to generate a multimodal standard behavior template library. The candidate multimodal data fusion module is used to collect time-series data of candidates' operational behavior, speech features, vehicle dynamics, and video images during the assessment process at a preset sampling period, and then fuse them after aligning them by timestamp to generate a multimodal driving behavior dataset. The multi-dimensional behavior deviation calculation module is used to compare the operation behavior time-series data, voice feature time-series data, and vehicle dynamic time-series data in the multimodal driving behavior dataset with the multimodal standard behavior template library item by item, calculate the behavior deviation quantification value under each dimension, and generate a multi-dimensional deviation feature set containing operation deviation features, voice deviation features, and dynamic deviation features. The driving basic score fusion module is used to calculate the operation standardization score, voice standardization score and driving stability score according to the multi-dimensional deviation feature set, and to perform weighted fusion of the operation standardization score, the voice standardization score and the driving stability score according to preset weights to generate a driving behavior basic score. The anomaly identification and comprehensive evaluation module is used to extract the coordinates of key points on the driver's face based on video image data in the multimodal driving behavior dataset and calculate the eyelid opening and closing time series. At the same time, it calculates the speed change rate and acceleration change rate based on the vehicle dynamic time series data. The eyelid opening and closing time series is compared with a preset fatigue threshold to identify fatigue driving behavior, and the speed change rate and acceleration change rate are compared with a preset aggressive threshold to identify aggressive driving behavior. When abnormal driving behavior is identified, an abnormal behavior warning log containing the abnormality type and corresponding deduction value is generated. Based on the abnormal behavior warning log, the basic driving behavior score is deducted repeatedly to generate a comprehensive driving behavior evaluation result.

9. A computer device comprising a memory and a processor, wherein the memory stores a computer program, characterized in that, When the processor executes the computer program, it implements the method of any one of claims 1 to 7.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by a processor, it implements the method of any one of claims 1 to 7.

Images