Intelligent evaluation system for virtual control of motor train unit
Through the multi-mode analysis model, the EMU simulated driving data is verified audio and video, which solves the problems of inefficiency and lack of judgment in the traditional training model, real-time correction and detailed analysis of the driving behavior of EMUs, and improves training efficiency and accuracy.
Patent Information
- Application Number
- CN202411008256.7
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-07-26
- Publication Date
- 2024-11-15
- Estimated Expiration
- 2044-07-26
AI Technical Summary
The traditional EMU simulated driving training model relies on manual monitoring, is inefficient and difficult to correct driving behaviors that do not meet the norms in real time. The judgment criteria for driving behaviors are not strict and the data analysis is not detailed enough.
Multimode analysis model is used to perform multimodal verification and real-time evaluation of behavioral action data and simulated operation data on audio and video, embed speech recognition model and video analysis model, improve multi-format data processing speed, and correct driving behaviors that do not comply with specifications in real time , and improve the judgment standards of driving behavior and the precision of data analysis.
It has achieved timely correction of driving behaviors that do not meet the norms, improved the standards for driving behavior judgments and the precision of simulated driving data analysis, and improved training efficiency and accuracy.
Smart Images

Figure CN118965106A_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to the technical field of intelligent evaluation, and in particular to an intelligent evaluation system for virtual operation of a motor vehicle group. Background Art
[0002] With the rapid development of high-speed rail, EMUs occupy an important position in the modern railway transportation system. In order to improve the safety and stability of EMUs and reduce the problems caused by human errors of drivers, the EMU virtual operation intelligent evaluation system has been developed. The system uses virtual reality technology and artificial intelligence technology to evaluate the driver's operation based on the simulation of real driving scenes, provide accurate feedback and guidance, thereby improving the driver's driving skills and operation level, and improving the safety and stability of EMUs.
[0003] In order to strengthen the awareness of EMU drivers to follow standardized operations and ensure the safe and smooth operation of trains, all railway bureaus and locomotive depots are equipped with advanced simulation driving systems. These systems exquisitely reproduce the various indication information of the HMI (human-machine interface) and DMI (driver's cab display) on the EMU control panel, and simulate the road conditions along the line and the dynamic driving conditions of the train in different scenarios in three dimensions on the high-definition display. During the trainee training process, the simulation system collects accurate data from the hardware feedback in real time for monitoring. Despite this, the traditional training model still faces bottlenecks: many trainees participate in long-term daily training at the same time (usually lasting about six to seven hours a day), relying on instructors to manually monitor the operations of each trainee to determine whether there are any violations. This practice not only makes it easy for instructors to make omissions due to long-term and high-intensity work, but also consumes precious resources and is inefficient. Therefore, it is urgent to explore a more efficient and accurate training evaluation solution to meet current challenges.
[0004] Patent No. CN2017114618060 discloses an on-board test evaluation terminal and test evaluation system, which can realize unified evaluation of subject two of the motor vehicle driver test and reduce the system and human factor risks during the test. The on-board test evaluation terminal includes an electronically connected power module, an audio module, a controller, an LCD display, a button module, a BD / GPS module, and a PLC signal acquisition module. The test evaluation system includes a communication-connected test background management system, a public security traffic management comprehensive application platform, a third-party on-board industrial computer, a third-party test center system, and an on-board evaluation terminal. At the same time, the above invention also provides a candidate identity authentication method.
[0005] Patent No. CN2020102967575 discloses an intelligent driving test evaluation method, which includes an information acquisition step, a data processing step and a vehicle-mounted control step. In the information acquisition step, the vehicle-mounted sensor is used to sense and collect the student's driving behavior data in real time, and the collected data is calculated and processed by the vehicle-mounted computing device in the data processing step, and compared with the driving behavior data in the preset driving test rule library to obtain a comparison result. According to the comparison result, it is determined whether the student's driving behavior is qualified. If it is unqualified, the vehicle-mounted computing device plans a safe path and controls the vehicle to the end of the safe path through the vehicle-mounted control device in the vehicle-mounted control step, thereby realizing the objectivity of the evaluation of test subjects that are easily affected by the examiner's personal subjective factors in the driving test, improving the fairness of the driving test, and realizing the safe control of the driving test vehicle, reducing traffic accidents caused by driving training and test vehicles, and ensuring road driving safety.
[0006] Although the above patents all judge the driving behavior during the vehicle simulation driving process, they are unable to correct the driving behavior that does not meet the standards in a timely manner, and the judgment criteria for certain driving behaviors are not strict, and the data analysis of the vehicle during the simulated driving process is not comprehensive and detailed enough. Summary of the invention
[0007] The purpose of the present invention is to provide an intelligent evaluation system for virtual operation of EMUs, which can perform multimodal verification and real-time evaluation of audio and video on behavioral action data and simulated operation data through a multimodal analysis model, and embed a speech recognition model and a video analysis model into the same large model to improve the speed of multi-format data processing, so as to promptly correct driving behaviors that do not meet the specifications, improve the judgment standards of driving behaviors, and at the same time improve the precision of EMU data analysis during simulated driving.
[0008] The present invention utilizes the following technical solutions:
[0009] An intelligent evaluation system for virtual operation of a train set includes an intelligent evaluation platform and several simulated driving terminals, wherein:
[0010] The simulated driving terminal is used to collect and pre-process all the behavioral action data of the driving trainee during the simulated driving process and the simulated operation data of the train;
[0011] Intelligent evaluation platform, which is used to evaluate the pre-processed behavior action data and simulation operation data according to the set multi-mode analysis model and store the evaluation results, and is also used to provide real-time feedback to driving trainees;
[0012] The simulated driving terminal includes an acquisition module, a processing module, a transmission module and a display module;
[0013] The acquisition module is used to obtain the behavior and action data of the driving trainee and the simulated operation data of the train;
[0014] A processing module, used for pre-processing the behavior action data and simulation operation data to construct a unique data block for each driver trainee;
[0015] The transceiver module is used to send the exclusive data block to the intelligent evaluation platform for evaluation and analysis, and receive the exclusive data block and evaluation feedback results of the evaluation and analysis completed from the intelligent evaluation platform according to the demonstration requirements;
[0016] The display module is used to display the practical training task guide to instruct the driver to perform simulated driving, and to display the exclusive data blocks and evaluation feedback results stored in the intelligent evaluation platform according to the demonstration requirements;
[0017] The intelligent evaluation platform includes a login module, an evaluation module and a storage module;
[0018] A login module is used to detect and verify the data transmission process of each simulated driving terminal;
[0019] The evaluation module is used to evaluate and feedback the exclusive data blocks transmitted by each simulated driving terminal to generate evaluation feedback results;
[0020] The storage module is used to store the exclusive data blocks and evaluation feedback results of the completed evaluation feedback in a database.
[0021] Preferably, the behavioral action data includes but is not limited to switch touches, voice commands, operating gestures and line of sight shifts; the simulated operation data includes but is not limited to the real-time simulated position of the trainee, the simulated trajectory of the trainee, the speed of the trainee, energy consumption statistics, energy consumption distribution, car temperature, car humidity and car air pressure; the acquisition module includes a motion capture unit and a data extraction unit; the motion capture unit uses a switch touch sensor to capture the number of switch touches of the driver during the simulated driving process, and uses a voice acquisition device to capture the voice commands of the driver during the simulated driving process; uses a video acquisition device to record the operating gestures and line of sight shifts of the driver during the simulated driving process; the data extraction unit uses a virtual sensor network to detect the simulated operation data generated by the trainee during the simulated driving process; the virtual sensor network includes but is not limited to digital inertial sensors, digital voltage sensors, digital current sensors, digital power sensors, digital temperature sensors, digital air pressure sensors and digital humidity sensors; the voice acquisition device can use a microphone; the video acquisition device can use a panoramic camera.
[0022] Preferably, the process of preprocessing the behavior action data and the simulation operation data by the processing module is as follows:
[0023] A: Use data segmentation algorithm based on data sources to divide action data and simulation operation data into several data blocks;
[0024] B: Aggregate the internal data of each data block according to a preset time interval, and divide each data block into several sub-data blocks according to the training number of the driving trainee; the preset time interval is hours or minutes;
[0025] C: Using clustering algorithm according to the training time of the driving trainees, the internal data of the sub-data block is classified;
[0026] D: Use data cleaning algorithms to clean abnormal data from the classified internal data; data cleaning algorithms include but are not limited to data deduplication algorithms, data filling algorithms, data conversion algorithms, and data simplification algorithms;
[0027] E: According to the training number of the driving trainee, the internal data of all sub-data blocks that have completed data cleaning are associated and normalized to construct an exclusive data block for each driving trainee.
[0028] Preferably, the transceiver module includes a channel construction unit and a protocol transmission unit; the channel construction unit uses a virtual hierarchical routing algorithm to construct a data transmission channel from each simulated driving terminal to the intelligent evaluation platform; the protocol transmission unit uses the CAN protocol combined with a parallel transmission algorithm and a cache retransmission mechanism to send the exclusive data blocks of the simulated driving terminal to the intelligent evaluation platform through the data transmission channel, and then selectively sends the stored exclusive data blocks to each simulated driving terminal.
[0029] Preferably, the display module includes an interactive unit and a display unit; the interactive unit uses an adjustment knob and a touch pad to selectively extract exclusive data blocks and training task guidelines in the intelligent evaluation platform; the display unit uses a display to display the selectively extracted exclusive data blocks and training task guidelines.
[0030] Preferably, the login module includes a terminal verification unit and a data verification unit; the terminal verification unit uses a chaotic verification algorithm to verify the login request code and device verification code of each simulated driving terminal; the data verification unit uses a time series analysis algorithm to verify the internal data of the exclusive data block according to the driver's training number and training time.
[0031] Preferably, the evaluation module includes a manipulation discrimination unit and a comprehensive feedback unit; the manipulation discrimination unit uses a preset multi-modal analysis model to perform a comprehensive analysis and comparison on the exclusive data block to generate a comprehensive analysis and comparison result; the comprehensive feedback unit uses a strategy feedback algorithm to generate a corresponding feedback strategy based on the comprehensive analysis and comparison result to form an evaluation feedback result.
[0032] Preferably, the multi-mode discrimination model includes a manipulation analysis layer, an operation reproduction layer, a feature association layer, an iterative training layer and a recognition and judgment layer; the manipulation analysis layer is used to extract and analyze the behavioral action data in the exclusive data block to obtain a manipulation action feature value set; the operation reproduction layer is used to extract and reproduce the simulated operation data in the exclusive data block to obtain a simulated reproduction feature value set; the feature association layer sequentially uses 4 3×3×1 convolution layers with a step size of 1, 2 5×5×1 convolution layers with a step size of 2 and 3 3×3×3 transposed convolution layers with a step size of 1 to extract features from the manipulation action feature value set and the simulation reproduction feature value set at the same time, respectively obtaining the manipulation action feature matrix and the simulation reproduction feature matrix, and then using the channel shuffle layer combined with the global nearest neighbor association algorithm to globally associate the manipulation action feature matrix and the simulation reproduction feature matrix to obtain a comprehensive feature matrix; the iterative training layer performs several iterative training on the comprehensive feature matrix based on the timestamp to obtain a comprehensive weight matrix; the iterative training layer It includes 2 3×3×3 convolutional layers with a stride of 2, 4 maximum pooling layers composed of 1×3×3 convolution and 3×3×1 convolution with a stride of 1, 3 average pooling layers composed of 3×3×3 spatial separation convolutions with a stride of 2, 4 residual blocks, 3 inverted residual blocks, 2 InceptionV2 blocks and a global attention mechanism; each residual block is connected in parallel with 1 InceptionV4 block, and each inverted residual block is connected in parallel with 2 series-connected InceptionV1 blocks. The global attention mechanism shapes the comprehensive feature matrix within the same timestamp in time series to form a continuous operation control chain of the trainee, thereby improving the simulation driving accuracy of the driving trainees; the recognition and judgment layer will collect and process the behavioral action data and the simulated operation data of the trainee in real time, after 2 global average pooling layers, 4 fully connected layers, LRelu activation function and MISH activation function, and then use the set over-phase to perform a comprehensive comparison of logical judgment to output a comprehensive analysis comparison result.
[0033] Preferably, the process of the manipulation analysis layer obtaining the manipulation action feature value set is as follows:
[0034] F1: Manipulate the parsing layer to extract the behavior action data in the exclusive data block;
[0035] F2: Divide the action data into audio data and video data according to the data format:
[0036]
[0037] Among them, Q1 represents audio data, R represents audio format, represents Hadamard inner product, BF() represents filter function, Q0 represents action data, Q2 represents video data, and M represents video format;
[0038] F3: parse and extract the voice commands in the audio data, and parse and extract the action commands in the video data: the action commands include switch touch, manipulation gestures and sight shift;
[0039] Q1=∑ i=1 (T i ⊙S i )+Q x (3)
[0040] Where i represents the voice command number, T i represents the i-th voice command, ⊙ represents the Hadamard outer product, S represents the over-phase, S i represents the over-phase corresponding to the i-th voice command, Q x Indicates redundant action data;
[0041]
[0042] Among them, j represents the action instruction number, Sth j represents the jth switch touch, Represents vector and, Mpg j represents the jth manipulation gesture, Sgz j represents the jth sight shift;
[0043] F4: Convert all extracted behavioral action data into corresponding feature values and construct them into a set of manipulation action feature values.
[0044] Preferably, the process of running the recurrence layer to obtain a set of simulated recurrence feature values is as follows:
[0045] G1: Run the recurrence layer to extract the simulation operation data in the dedicated data block;
[0046] G2: Divide the simulation operation data into operation data and environment data according to data attributes;
[0047] The data attribute is the place where the current data is collected from the train; the operation data includes but is not limited to the real-time simulated position of the train, the simulated trajectory of the train, the speed of the train, the energy consumption statistics and the energy consumption distribution; the environmental data includes but is not limited to the temperature, humidity and air pressure of the carriage;
[0048] G3: During the simulated driving process, the relationship between operational data and environmental data is:
[0049] The real-time simulated position of the train is continuously mapped f() to generate the simulated trajectory of the train: Trajectory = f(Position); the energy consumption statistics Energy consumption is proportional to the speed of the train g(): Energy consumption = g(Velocity); the energy consumption distribution Energy distribution is in a discrete sampling relationship with the speed of the train h(): Energy distribution = h(Velocity); the compartment temperature Temperature, compartment humidity Humidity and compartment pressure Pressure are in a k()-dimensional relationship with time time and the real-time simulated position of the train Position:
[0050] Temperature,Humidity,Pressure=k(time,Position);
[0051] Energy consumption statistics, energy consumption distribution and cabin temperature have a linear regression relationship:
[0052] Energy consumption=β0+β1*Temperature+β2*Energy distribution+ε(5)
[0053] Among them, β0 represents the first regression coefficient, β1 represents the second regression coefficient, β2 represents the third regression coefficient, and ε represents the error coefficient;
[0054] G4: Convert all extracted simulation run data into corresponding eigenvalues and construct them into a set of simulation recurrence eigenvalues.
[0055] The present invention uses a multi-mode analysis model to perform multi-modal verification and real-time evaluation of audio and video on behavioral action data and simulated operation data, embeds the speech recognition model and the video analysis model into the same large model, improves the speed of multi-format data processing, and promptly corrects driving behaviors that do not meet the specifications, improves the judgment standards for driving behaviors, and improves the precision of vehicle data analysis during simulated driving. BRIEF DESCRIPTION OF THE DRAWINGS
[0056] In order to more clearly illustrate the technical solutions in the embodiments of the present invention or related technologies, the drawings required for use in the embodiments or related technical descriptions are briefly introduced below. Obviously, the drawings described below are only embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on the provided drawings without paying creative work.
[0057] Figure 1 This is the principle block diagram of the intelligent evaluation system;
[0058] Figure 2 This is the workflow diagram of the intelligent evaluation system. DETAILED DESCRIPTION
[0059] The present invention is described in detail below with reference to the accompanying drawings and embodiments:
[0060] like Figure 1-Figure 2 As shown, the intelligent evaluation system for virtual operation of a train set of the present invention comprises an intelligent evaluation platform and a plurality of simulated driving terminals, wherein:
[0061] The simulated driving terminal is used to collect and pre-process all the behavioral action data of the driving trainee during the simulated driving process and the simulated operation data of the train;
[0062] Intelligent evaluation platform, which is used to evaluate the pre-processed behavior action data and simulation operation data according to the set multi-mode analysis model and store the evaluation results, and is also used to provide real-time feedback to driving trainees;
[0063] The simulated driving terminal includes an acquisition module, a processing module, a transmission module and a display module;
[0064] The acquisition module is used to obtain the behavior and action data of the driving trainee and the simulated operation data of the train;
[0065] In the present invention, the behavioral action data includes but is not limited to switch touch, voice command, manipulation gesture and line of sight shift; the simulation operation data includes but is not limited to the real-time simulation position of the motor vehicle, the simulation trajectory of the motor vehicle, the speed of the motor vehicle, the energy consumption statistics, the energy consumption distribution, the compartment temperature, the compartment humidity and the compartment air pressure;
[0066] The acquisition module includes a motion capture unit and a data extraction unit; the motion capture unit uses a switch touch sensor to capture the switch touch of the driver during the simulated driving process, and uses a voice acquisition device to collect the voice instructions of the driver during the simulated driving process; the video acquisition device is used to record the driver's operating gestures and line of sight shift during the simulated driving process; the data extraction unit uses a virtual sensor network to detect the simulated operation data generated by the motor vehicle during the simulated driving process;
[0067] The virtual sensor network includes but is not limited to digital inertial sensors, digital voltage sensors, digital current sensors, digital power sensors, digital temperature sensors, digital air pressure sensors and digital humidity sensors; the voice acquisition device may use a microphone; the video acquisition device may use a panoramic camera;
[0068] A processing module, used for pre-processing the behavior action data and simulation operation data to construct a unique data block for each driver trainee;
[0069] In the present invention, the process of preprocessing the behavior action data and the simulation operation data by the processing module is as follows:
[0070] A: Use data segmentation algorithm based on data sources to divide action data and simulation operation data into several data blocks;
[0071] B: Aggregate the internal data of each data block according to a preset time interval, and divide each data block into several sub-data blocks according to the training number of the driving trainee; the preset time interval is hours or minutes;
[0072] C: Using clustering algorithm according to the training time of the driving trainees, the internal data of the sub-data block is classified;
[0073] D: Use data cleaning algorithms to clean abnormal data from the classified internal data; data cleaning algorithms include but are not limited to data deduplication algorithms, data filling algorithms, data conversion algorithms, and data simplification algorithms;
[0074] E: According to the training number of the driving trainee, the internal data of all sub-data blocks that have completed data cleaning are associated and normalized to construct an exclusive data block for each driving trainee;
[0075] The transceiver module is used to send the exclusive data block to the intelligent evaluation platform for evaluation and analysis, and receive the exclusive data block and evaluation feedback results of the evaluation and analysis completed from the intelligent evaluation platform according to the demonstration requirements;
[0076] In the present invention, the transceiver module includes a channel construction unit and a protocol transmission unit; the channel construction unit uses a virtual hierarchical routing algorithm to construct a data transmission channel from each simulated driving terminal to the intelligent evaluation platform; the protocol transmission unit uses the CAN protocol combined with a parallel transmission algorithm and a cache retransmission mechanism to send the exclusive data block of the simulated driving terminal to the intelligent evaluation platform through the data transmission channel, and then selectively sends the stored exclusive data block to each simulated driving terminal;
[0077] The display module is used to display the practical training task guide to instruct the driver to perform simulated driving, and to display the exclusive data blocks and evaluation feedback results stored in the intelligent evaluation platform according to the demonstration requirements;
[0078] In the present invention, the display module includes an interaction unit and a display unit; the interaction unit uses an adjustment knob and a touch panel to selectively extract the exclusive data blocks and the practical training task guide in the intelligent evaluation platform; the display unit uses a display to display the selectively extracted exclusive data blocks and the practical training task guide;
[0079] The intelligent evaluation platform includes a login module, an evaluation module and a storage module;
[0080] A login module is used to detect and verify the data transmission process of each simulated driving terminal;
[0081] In the present invention, the login module includes a terminal verification unit and a data verification unit; the terminal verification unit uses a chaotic verification algorithm to verify the login request code and the device verification code of each simulated driving terminal; the data verification unit uses a time series analysis algorithm to verify the internal data of the exclusive data block according to the training number and training time of the driving trainee;
[0082] In this embodiment, the login request code refers to a login request code that the simulated driving terminal will send to the intelligent evaluation platform when the driver logs in, which is used to identify the uniqueness of this login request. After receiving the login request code, the intelligent evaluation platform will verify the legitimacy of the request and perform login-related security checks, such as the correctness of the account password, the trust of the login device, etc.;
[0083] The device verification code means that in order to further enhance the security of the driver's login, the intelligent evaluation platform will also verify the device used by the driver to ensure the legitimacy of the login device. When a user logs in with a new device, the intelligent evaluation platform will send a device verification code to the simulated driving terminal, and the simulated driving terminal needs to enter the code into the login interface to verify and authenticate the device information;
[0084] The evaluation module is used to evaluate and feedback the exclusive data blocks transmitted by each simulated driving terminal to generate evaluation feedback results;
[0085] In the present invention, the evaluation module includes a manipulation discrimination unit and a comprehensive feedback unit; the manipulation discrimination unit uses a preset multi-mode discrimination model to perform a comprehensive analysis and comparison on the exclusive data block to generate a comprehensive analysis and comparison result; the comprehensive feedback unit uses a strategy feedback algorithm to generate a corresponding feedback strategy according to the comprehensive analysis and comparison result to form an evaluation feedback result;
[0086] In this embodiment, the multi-mode discrimination model includes a manipulation analysis layer, an operation reproduction layer, a feature association layer, an iterative training layer, and a recognition judgment layer;
[0087] The manipulation analysis layer is used to extract and analyze the behavior action data in the exclusive data block to obtain a set of manipulation action feature values;
[0088] In this embodiment, the process of the manipulation analysis layer obtaining the manipulation action feature value set is as follows:
[0089] F1: Manipulate the parsing layer to extract the behavior action data in the exclusive data block;
[0090] F2: Divide the action data into audio data and video data according to the data format:
[0091]
[0092] Among them, Q1 represents audio data, R represents audio format, represents Hadamard inner product, BF() represents filter function, Q0 represents action data, Q2 represents video data, and M represents video format;
[0093] F3: parse and extract the voice commands in the audio data, and parse and extract the action commands in the video data: the action commands include switch touch, manipulation gestures and sight shift;
[0094] Q1=∑ i=1 (T i ⊙S i )+Q x (3)
[0095] Where i represents the voice command number, T i represents the i-th voice command, ⊙ represents the Hadamard outer product, S represents the over-phase, S i represents the over-phase corresponding to the i-th voice command, Q x Indicates redundant action data;
[0096]
[0097] Among them, j represents the action instruction number, Sth j represents the jth switch touch, Represents vector and, Mpg j represents the jth manipulation gesture, Sgz j represents the jth sight shift;
[0098] F4: Convert all extracted behavior action data into corresponding feature values and construct them into a set of manipulation action feature values;
[0099] The operation reproduction layer is used to extract and reproduce the simulation operation data in the exclusive data block to obtain a set of simulation reproduction feature values;
[0100] In this embodiment, the process of running the recurrence layer to obtain a set of simulated recurrence feature values is as follows:
[0101] G1: Run the recurrence layer to extract the simulation operation data in the dedicated data block;
[0102] G2: Divide the simulation operation data into operation data and environment data according to data attributes;
[0103] The data attributes are the data sources from the process of simulated driving of the train; the operation data include but are not limited to the real-time simulated position of the train, the simulated trajectory of the train, the speed of the train, the energy consumption statistics and the energy consumption distribution; the environmental data include but are not limited to the temperature, humidity and air pressure of the carriage;
[0104] G3: During the simulated driving process, the relationship between operational data and environmental data is:
[0105] The real-time simulated position of the train is continuously mapped f() to generate the simulated trajectory of the train: Trajectory = f(Position); the energy consumption statistics Energy consumption is proportional to the speed of the train g(): Energy consumption = g(Velocity); the energy consumption distribution Energy distribution is in a discrete sampling relationship with the speed of the train h(): Energy distribution = h(Velocity); the compartment temperature Temperature, compartment humidity Humidity and compartment pressure Pressure are in a k()-dimensional relationship with time time and the real-time simulated position of the train Position:
[0106] Temperature,Humidity,Pressure=k(time,Position);
[0107] Energy consumption statistics, energy consumption distribution and cabin temperature have a linear regression relationship:
[0108] Energy consumption =
[0109] β0+β1*Temperature+β2*Energy distribution+ε(5)
[0110] Among them, β0 represents the first regression coefficient, β1 represents the second regression coefficient, β2 represents the third regression coefficient, and ε represents the error coefficient;
[0111] G4: Convert all extracted simulation operation data into corresponding eigenvalues and construct them into a set of simulation recurrence eigenvalues;
[0112] The feature association layer uses four 3×3×1 convolutional layers with a step size of 1, two 5×5×1 convolutional layers with a step size of 2, and three 3×3×3 transposed convolutional layers with a step size of 1 in sequence to extract features from the manipulation action feature value set and the simulation reproduction feature value set at the same time, and obtain the manipulation action feature matrix and the simulation reproduction feature matrix respectively. Then, the channel shuffle layer is combined with the global nearest neighbor association algorithm to globally associate the manipulation action feature matrix and the simulation reproduction feature matrix to obtain a comprehensive feature matrix.
[0113] The iterative training layer performs several iterative training on the comprehensive feature matrix based on the timestamp to obtain the comprehensive weight matrix; the iterative training layer includes 2 3×3×3 convolution layers with a step size of 2, 4 maximum pooling layers composed of 1×3×3 convolution and 3×3×1 convolution with a step size of 1, 3 average pooling layers composed of 3×3×3 spatial separation convolutions with a step size of 2, 4 residual blocks, 3 inverted residual blocks, 2 InceptionV2 blocks and a global attention mechanism; each residual block is connected in parallel with 1 InceptionV4 block, and each inverted residual block is connected in parallel with 2 series-connected InceptionV1 blocks. The global attention mechanism shapes the comprehensive feature matrix within the same timestamp in time series to form a continuous operation control chain of the trainee, thereby improving the simulated driving accuracy of the driving trainees;
[0114] The recognition and evaluation layer collects and processes the behavioral action data and the simulated operation data of the train in real time, passes them through two global average pooling layers, four fully connected layers, LRelu activation function and MISH activation function, and then uses the set transition phase to perform a comprehensive comparison of logical judgment to output a comprehensive analysis comparison result;
[0115] In this embodiment, the working principle of the multi-mode discrimination model is:
[0116] First, the manipulation parsing layer extracts the behavior action data in the exclusive data block, divides the behavior action data into audio data and video data according to the data format, parses and extracts the voice commands in the audio data, and parses and extracts the switch touch, manipulation gesture and line of sight transfer in the video data; converts all the extracted behavior action data into corresponding feature values, and constructs them into a set of manipulation action feature values; at the same time, the operation reproduction layer extracts the simulation operation data in the exclusive data block; divides the simulation operation data into operation data and environmental data according to data attributes; in the simulation driving process, the relationship between operation data and environmental data is: the real-time simulation position of the EMU is continuously mapped to generate the EMU driving simulation trajectory, the energy consumption statistics is proportional to the EMU driving speed, the energy consumption distribution is in a discrete sampling relationship with the EMU driving speed, the car temperature, car humidity and car air pressure are diversified with time and the real-time simulation position of the EMU, and the energy consumption statistics, energy consumption distribution and car temperature have a linear regression relationship; all the extracted simulation operation data are converted into corresponding feature values, and constructed into a set of simulation reproduction feature values;
[0117] Then, the feature association layer uses four 3×3×1 convolutional layers with a step size of 1, two 5×5×1 convolutional layers with a step size of 2, and three 3×3×3 transposed convolutional layers with a step size of 1 in sequence to extract features from the manipulation action feature value set and the simulation reproduction feature value set at the same time, and obtain the manipulation action feature matrix and the simulation reproduction feature matrix respectively. Then, the channel shuffle layer is combined with the global nearest neighbor association algorithm to globally associate the manipulation action feature matrix and the simulation reproduction feature matrix to obtain a comprehensive feature matrix.
[0118] Subsequently, the iterative training layer performs several iterative training on the comprehensive feature matrix based on the timestamp to obtain the comprehensive weight matrix; the iterative training layer includes 2 3×3×3 convolution layers with a step size of 2, 4 maximum pooling layers composed of 1×3×3 convolution and 3×3×1 convolution with a step size of 1, 3 average pooling layers composed of 3×3×3 spatial separation convolutions with a step size of 2, 4 residual blocks, 3 inverted residual blocks, 2 InceptionV2 blocks and a global attention mechanism; each residual block is connected in parallel with 1 InceptionV4 block, and each inverted residual block is connected in parallel with 2 series-connected InceptionV1 blocks. The global attention mechanism shapes the comprehensive feature matrix within the same timestamp in time series to form a continuous operation control chain of the trainee, thereby improving the simulated driving accuracy of the driving trainees;
[0119] Finally, the recognition and evaluation layer will collect and process the behavioral action data and the simulated operation data of the EMU in real time, and after passing them through 2 global average pooling layers, 4 fully connected layers, LRelu activation function and MISH activation function, it will use the set transition phase to perform a comprehensive comparison of logical judgment to output a comprehensive analysis and comparison result.
[0120] A storage module is used to store the dedicated data blocks and evaluation feedback results of the completed evaluation feedback in a database;
[0121] In this embodiment, the database includes but is not limited to a metadata database and a relational database;
[0122] In the present invention, the switch touch sensor, voice acquisition device, video acquisition device, virtual sensor network, data blocking algorithm, clustering algorithm, data cleaning algorithm, virtual hierarchical routing algorithm, CAN protocol, parallel transmission algorithm, cache retransmission mechanism, adjustment knob, touch pad, display, chaos check algorithm, time series analysis algorithm and strategy feedback algorithm are all existing technologies and will not be described in detail here.
[0123] Example:
[0124] The intelligent evaluation system first performs a self-check procedure to ensure that all hardware components (touch detection modules such as touch screens, voice input modules such as voice acquisition devices, video input modules such as panoramic cameras, etc.) and software modules (voice recognition engines, image processing algorithms, over-phase logic judgment, etc.) are operating normally; load multi-modal analysis models into memory, which are pre-trained and can recognize specific voice commands and driving operation behaviors. At the same time, load predefined standard process models for key operations such as over-phase; and switch the intelligent evaluation system to training mode. The display unit uses the display to display the selectively extracted training task guide, waiting for the driver to start operating;
[0125] Then, the acquisition module of the simulated driving end uses a motion capture unit to use a switch touch sensor to capture the switch touch of the driver during the simulated driving process, and uses a voice acquisition device to collect the voice commands of the driver during the simulated driving process; uses a video acquisition device to record the driver's operating gestures and line of sight shifts during the simulated driving process; uses a data extraction unit to use a virtual sensor network to detect the simulated operation data generated by the motor vehicle during the simulated driving process; the behavioral action data includes but is not limited to switch touch, voice command, operating gesture and line of sight shift; the simulated operation data includes but is not limited to the real-time simulated position of the motor vehicle, the simulated trajectory of the motor vehicle, the driving speed of the motor vehicle, energy consumption statistics, energy consumption distribution, car temperature, car humidity and car air pressure;
[0126] The processing module pre-processes the behavior action data and the simulation operation data, and divides the behavior action data and the simulation operation data into several data blocks according to the data source by using the data block algorithm; the internal data of each data block is aggregated according to the preset time interval, and each data block is divided into several sub-data blocks according to the practical training number of the driver trainee; the preset time interval is hours or minutes; the internal data of the sub-data blocks are classified by using the clustering algorithm according to the training time of the driver trainee; the abnormal data is cleaned by using the data cleaning algorithm for the internal data that has completed the classification; the data cleaning algorithm includes but is not limited to the data deduplication algorithm, the data filling algorithm, the data conversion algorithm and the data simplification algorithm; according to the practical training number of the driver trainee, the internal data of all the sub-data blocks that have completed the data cleaning are associated and normalized to construct the exclusive data block of each driver trainee;
[0127] The channel construction unit of the transceiver module uses a virtual hierarchical routing algorithm to construct a data transmission channel from each simulated driving terminal to the intelligent evaluation platform; the protocol transmission unit uses the CAN protocol combined with a parallel transmission algorithm and a cache retransmission mechanism to send the exclusive data block of the simulated driving terminal to the intelligent evaluation platform through the data transmission channel, and then selectively sends the stored exclusive data block to each simulated driving terminal; the interactive unit of the display module uses an adjustment knob and a touch pad to selectively extract the exclusive data blocks and training task guides in the intelligent evaluation platform; the terminal verification unit of the login module uses a chaotic verification algorithm to verify the login request code and device verification code of each simulated driving terminal; the data verification unit uses a time series analysis algorithm to verify the internal data of the exclusive data block according to the training number and training time of the driver; the manipulation discrimination unit of the evaluation module uses a preset multi-mode discrimination model to perform a comprehensive analysis and comparison of the exclusive data block to generate a comprehensive analysis and comparison result:
[0128] The voice commands issued by the driving trainee during the operation are captured in real time. The simulated driving end pre-processes the collected voice signals, extracts its feature vectors, and matches them with the preset voice models related to the over-phase operation. For example, it recognizes keywords such as "prepare for over-phase"; if the recognized voice information does not match the standard instructions for over-phase operation, the system automatically records the deduction points and prompts the trainee with the correct operation instructions through voice or screen;
[0129] For video data, the intelligent evaluation platform performs grayscale processing on the video streams before and after 10 seconds to reduce color interference, and uses median filtering and other methods to eliminate noise in the image to ensure the accuracy of subsequent analysis. The frame difference method is used to analyze the processed video sequence, identify changes in key operations such as hand gestures and fist clenching, and compare them with standard operating procedures; if it is found that the driver's operation does not conform to the standard operating procedures (such as not paying attention to the hand gestures), the system will record the corresponding deduction points, and point out the specific errors through visual or voice prompts to guide the students to operate correctly;
[0130] Then, the comprehensive feedback unit uses the strategy feedback algorithm to generate the corresponding feedback strategy according to the comprehensive analysis and comparison results to form the evaluation feedback results;
[0131] Finally, the storage module is used to store the exclusive data blocks and evaluation feedback results of the evaluation feedback in the database.
Claims
1. An intelligent evaluation system for virtual operation of EMU, characterized by: It includes an intelligent evaluation platform and several simulated driving terminals, including: The simulated driving terminal is used to collect and pre-process all the behavioral action data of the driving trainee during the simulated driving process and the simulated operation data of the train; Intelligent evaluation platform, which is used to evaluate the pre-processed behavior action data and simulation operation data according to the set multi-mode analysis model and store the evaluation results, and is also used to provide real-time feedback to driving trainees; The simulated driving terminal includes an acquisition module, a processing module, a transmission module and a display module; The acquisition module is used to obtain the behavior and action data of the driving trainee and the simulated operation data of the train; A processing module, used for pre-processing the behavior action data and simulation operation data to construct a unique data block for each driver trainee; The transceiver module is used to send the exclusive data block to the intelligent evaluation platform for evaluation and analysis, and receive the exclusive data block and evaluation feedback results of the evaluation and analysis completed from the intelligent evaluation platform according to the demonstration requirements; The display module is used to display the practical training task guide to instruct the driver to perform simulated driving, and to display the exclusive data blocks and evaluation feedback results stored in the intelligent evaluation platform according to the demonstration requirements; The intelligent evaluation platform includes a login module, an evaluation module and a storage module; A login module is used to detect and verify the data transmission process of each simulated driving terminal; The evaluation module is used to evaluate and feedback the exclusive data blocks transmitted by each simulated driving terminal to generate evaluation feedback results; The storage module is used to store the exclusive data blocks and evaluation feedback results of the completed evaluation feedback in a database.
2. The EMU virtual operation intelligent evaluation system according to claim 1 is characterized by: The behavioral action data include but are not limited to switch touch, voice command, manipulation gesture and line of sight shift; the simulation operation data include but are not limited to the real-time simulation position of the train, the simulation trajectory of the train, the speed of the train, energy consumption statistics, energy consumption distribution, compartment temperature, compartment humidity and compartment air pressure; the acquisition module includes a motion capture unit and a data extraction unit; The motion capture unit uses a switch touch sensor to capture the number of switch touches made by the driver trainee during the simulated driving process, and uses a voice acquisition device to capture the voice commands of the driver trainee during the simulated driving process; Use video acquisition equipment to record the driving student's control gestures and line of sight shifts during simulated driving; The data extraction unit uses a virtual sensor network to detect the simulated operation data generated by the motor vehicle during the simulated driving process.
3. The EMU virtual operation intelligent evaluation system according to claim 1 is characterized by: The process of the processing module preprocessing the behavior action data and the simulation operation data is as follows: A: Use data segmentation algorithm based on data sources to divide action data and simulation operation data into several data blocks; B: Aggregate the internal data of each data block according to a preset time interval, and divide each data block into several sub-data blocks according to the training number of the driving trainee; the preset time interval is hours or minutes; C: Using clustering algorithm according to the training time of the driving trainees, the internal data of the sub-data block is classified; D: Use data cleaning algorithms to clean abnormal data from the classified internal data; data cleaning algorithms include but are not limited to data deduplication algorithms, data filling algorithms, data conversion algorithms, and data simplification algorithms; E: According to the training number of the driving trainee, the internal data of all sub-data blocks that have completed data cleaning are associated and normalized to construct an exclusive data block for each driving trainee.
4. The EMU virtual operation intelligent evaluation system according to claim 1 is characterized by: The transceiver module includes a channel construction unit and a protocol transmission unit; the channel construction unit uses a virtual level routing algorithm to construct a data transmission channel from each simulated driving terminal to the intelligent evaluation platform; The protocol transmission unit uses the CAN protocol combined with a parallel transmission algorithm and a cache retransmission mechanism to send the exclusive data blocks of the simulated driving end to the intelligent evaluation platform through the data transmission channel, and then selectively sends the stored exclusive data blocks to each simulated driving end.
5. The EMU virtual operation intelligent evaluation system according to claim 1 is characterized by: The display module includes an interactive unit and a display unit; the interactive unit uses an adjustment knob and a touch pad to selectively extract exclusive data blocks and training task guidelines in the intelligent evaluation platform; the display unit uses a display to display the selectively extracted exclusive data blocks and training task guidelines.
6. The EMU virtual operation intelligent evaluation system according to claim 1 or 3, characterized in that: The login module includes a terminal verification unit and a data verification unit; the terminal verification unit uses a chaotic verification algorithm to verify the login request code and device verification code of each simulated driving terminal; the data verification unit uses a time series analysis algorithm to verify the internal data of the exclusive data block according to the training number and training time of the driving trainee.
7. The EMU virtual operation intelligent evaluation system according to claim 1 is characterized by: The evaluation module includes a manipulation discrimination unit and a comprehensive feedback unit; the manipulation discrimination unit uses a preset multi-mode discrimination model to perform a comprehensive analysis and comparison on the exclusive data block to generate a comprehensive analysis and comparison result; The comprehensive feedback unit uses a strategy feedback algorithm to generate a corresponding feedback strategy based on the comprehensive analysis and comparison results to form an evaluation feedback result.
8. The EMU virtual operation intelligent evaluation system according to claim 7 is characterized by: The multi-mode discrimination model includes a manipulation analysis layer, an operation reproduction layer, a feature association layer, an iterative training layer and an identification and judgment layer; the manipulation analysis layer is used to extract and analyze the behavior action data in the exclusive data block to obtain a set of manipulation action feature values; the operation reproduction layer is used to extract and reproduce the simulation operation data in the exclusive data block to obtain a set of simulation reproduction feature values; The feature association layer uses four 3×3×1 convolutional layers with a step size of 1, two 5×5×1 convolutional layers with a step size of 2, and three 3×3×3 transposed convolutional layers with a step size of 1 in sequence to extract features from the manipulation action feature value set and the simulation reproduction feature value set at the same time, and obtain the manipulation action feature matrix and the simulation reproduction feature matrix respectively. Then, the channel shuffle layer is combined with the global nearest neighbor association algorithm to globally associate the manipulation action feature matrix and the simulation reproduction feature matrix to obtain a comprehensive feature matrix. The iterative training layer performs several iterative training on the comprehensive feature matrix based on the timestamp to obtain the comprehensive weight matrix; the iterative training layer includes 2 3×3×3 convolution layers with a step size of 2, 4 maximum pooling layers composed of 1×3×3 convolution and 3×3×1 convolution with a step size of 1, 3 average pooling layers composed of 3×3×3 spatial separation convolutions with a step size of 2, 4 residual blocks, 3 inverted residual blocks, 2 InceptionV2 blocks and a global attention mechanism; each residual block is connected in parallel with an InceptionV4 block, and each inverted residual block is connected in parallel with an InceptionV4 block. The difference block is connected in parallel with two serially connected InceptionV1 blocks, and the global attention mechanism shapes the comprehensive feature matrix within the same timestamp in time series to form a continuous operation control chain of the train, thereby improving the simulation driving accuracy of the driving trainees; the recognition and judgment layer collects and processes the behavioral action data and the simulated operation data of the train in real time, passes through two global average pooling layers, four fully connected layers, LRelu activation function and MISH activation function, and uses the set over-phase to perform a full-scale comparison of logical judgment to output a full-scale analysis and comparison result.
9. The EMU virtual operation intelligent evaluation system according to claim 8 is characterized by: The process of the manipulation analysis layer obtaining the manipulation action feature value set is as follows: F1: Manipulate the parsing layer to extract the behavior action data in the exclusive data block; F2: Divide the action data into audio data and video data according to the data format: Among them, Q1 represents audio data, R represents audio format, represents Hadamard inner product, BF() represents filter function, Q0 represents action data, Q2 represents video data, and M represents video format; F3: parse and extract the voice commands in the audio data, and parse and extract the action commands in the video data: the action commands include switch touch, manipulation gestures and sight shift; Q1=∑ i=1 (T i ⊙S i )+Q x (3) Where i represents the voice command number, T i represents the i-th voice command, ⊙ represents the Hadamard outer product, S represents the over-phase, S i represents the over-phase corresponding to the i-th voice command, Q x Indicates redundant action data; Among them, j represents the action instruction number, Sth j represents the jth switch touch, Represents vector and, Mpg j represents the jth manipulation gesture, Sgz j represents the jth sight shift; F4: Convert all extracted behavioral action data into corresponding feature values and construct them into a set of manipulation action feature values.
10. The EMU virtual operation intelligent evaluation system according to claim 8 is characterized by: The process of running the recurrence layer to obtain a set of simulated recurrence feature values is as follows: G1: Run the recurrence layer to extract the simulation operation data in the dedicated data block; G2: Divide the simulation operation data into operation data and environment data according to data attributes; The data attribute is the place where the current data is collected from the train; the operation data includes but is not limited to the real-time simulated position of the train, the simulated trajectory of the train, the speed of the train, the energy consumption statistics and the energy consumption distribution; the environmental data includes but is not limited to the temperature, humidity and air pressure of the carriage; G3: During the simulated driving process, the relationship between operational data and environmental data is: The real-time simulated position of the train is continuously mapped f() to generate the simulated trajectory of the train: Trajectory = f(Position); the energy consumption statistics Energy consumption is proportional to the speed of the train g(): Energy consumption = g(Velocity); the energy consumption distribution Energy distribution is in a discrete sampling relationship with the speed of the train h(): Energy distribution = h(Velocity); the compartment temperature Temperature, compartment humidity Humidity and compartment pressure Pressure are in a k()-dimensional relationship with time time and the real-time simulated position of the train Position: Temperature, Humidity, Pressure=k(time, Position); Energy consumption statistics, energy consumption distribution and cabin temperature have a linear regression relationship: Energy consumption=β0+β1*Temperature+β2*Energy distribution+ε(5) Among them, β0 represents the first regression coefficient, β1 represents the second regression coefficient, β2 represents the third regression coefficient, and ε represents the error coefficient; G4: Convert all extracted simulation run data into corresponding eigenvalues and construct them into a set of simulation recurrence eigenvalues.
Citation Information
Patent Citations
Intelligent immersive high-speed train virtual driving behavior evaluation method and simulation system
CN106373453A
Driving standardization monitoring system for locomotive driver
CN109189019A
Teaching scoring method and system based on simulated driving
CN113658023A
Cited By
Motor train unit driver state identification method and system
CN121767969A
