Driving skill training method and system combined with virtual reality

Abstract

The invention discloses a driving skill training method and system combined with virtual reality, and belongs to the technical field of driving simulation, and the method comprises the steps: generating a plurality of driving scenes according to a virtual reality driving simulator; the training user performs driving simulation in multiple driving scenes according to the VR wearable device to obtain multiple groups of driving simulation data sets; constructing a driving operation scoring model; inputting the plurality of groups of driving simulation data sets into a driving operation scoring model, analyzing each group of driving simulation data sets according to the driving operation scoring model, and outputting a plurality of driving operation scoring results; and generating a driving skill training report according to the plurality of driving operation scoring results. The technical problem that in the prior art, driving skill training lacks personalized and datamation evaluation, so that trainees cannot obtain targeted training and feedback according to own performance is solved.

Description

Technical Field

[0001] The present invention relates to the technical field of driving simulation, and in particular to a driving skill training method and system combined with virtual reality. Background Art

[0002] With the rapid development of science and technology, virtual reality (VR) technology has been widely used in various fields, especially in the field of education and training. With its immersiveness and interactivity, virtual reality has become an important innovative tool. Especially in driving skills training, traditional teaching methods often rely on actual road driving or driving simulators, but these methods have certain limitations, such as high cost, limited training venues, and uncontrollable safety risks. With the maturity of virtual reality technology, driving skills training combined with virtual reality can not only simulate a variety of real driving environments, but also conduct repeated practice in a safe and controllable virtual environment, providing students with a more flexible and efficient learning method.

[0003] At present, traditional driving training relies on real car operation and simple driving simulators, and the students' learning progress and skill evaluation usually rely on the subjective judgment of the instructor. Although some driving simulators can provide certain driving scene simulations, the following problems still exist: First, traditional driving simulation training lacks a comprehensive and objective evaluation of students' driving operations; second, the training content lacks personalization, and students often cannot effectively train their weak links; third, the training environment and training progress are difficult to adjust in real time, and it is difficult to ensure that students are trained in the best learning state. Therefore, there is an urgent need for a more scientific and efficient driving skills training method that can combine virtual reality technology and data analysis to provide students with customized training programs and improve training results through real-time feedback and evaluation. Summary of the invention

[0004] This application provides a driving skills training method and system combined with virtual reality, aiming to solve the technical problem in the prior art that driving skills training lacks personalized and data-based evaluation, resulting in trainees being unable to obtain targeted training and feedback based on their own performance.

[0005] In view of the above problems, the present application provides a driving skills training method and system combined with virtual reality.

[0006] In a first aspect disclosed in the present application, a driving skill training method combined with virtual reality is provided, the method comprising connecting a virtual reality driving simulator, generating multiple driving scenarios according to the virtual reality driving simulator, wherein the virtual reality driving simulator comprises a VR wearable device; training users to perform driving simulations in the multiple driving scenarios according to the VR wearable device, and obtaining multiple groups of driving simulation data sets, wherein each group of driving simulation data sets corresponds to a driving scenario; constructing a driving operation scoring model, wherein the driving operation scoring model is connected to the virtual reality driving simulator; inputting the multiple groups of driving simulation data sets into the driving operation scoring model, analyzing each group of driving simulation data sets according to the driving operation scoring model, and outputting multiple driving operation scoring results; and generating a driving skill training report according to the multiple driving operation scoring results.

[0007] Another aspect disclosed in the present application provides a driving skill training system combined with virtual reality, the system including a driving scenario generation module, which is used to connect to a virtual reality driving simulator and generate multiple driving scenarios according to the virtual reality driving simulator, wherein the virtual reality driving simulator includes a VR wearable device; a simulation data set acquisition module, which is used to train users to perform driving simulations in the multiple driving scenarios according to the VR wearable device, and acquire multiple groups of driving simulation data sets, wherein each group of driving simulation data sets corresponds to a driving scenario; a scoring model construction module, which is used to construct a driving operation scoring model, wherein the driving operation scoring model is connected to the virtual reality driving simulator; a scoring result output module, which is used to input the multiple groups of driving simulation data sets into the driving operation scoring model, analyze each group of driving simulation data sets according to the driving operation scoring model, and output multiple driving operation scoring results; a training report generation module, which is used to generate a driving skill training report according to the multiple driving operation scoring results.

[0008] One or more technical solutions provided in this application have at least the following technical effects or advantages:

[0009] The driving skills training method combined with virtual reality is adopted, in which multiple driving scenarios are generated through a virtual reality driving simulator, and driving simulation data sets are obtained based on the students' performance in different scenarios. The data sets are analyzed and scored through a driving operation scoring model, which solves the problem of lack of personalized and data-based evaluation in driving skills training in the existing technology, resulting in students being unable to receive targeted training and feedback based on their own performance. This technical solution makes the training process more personalized through precise scoring and data analysis, and can provide customized training plans and feedback based on the students' performance, thereby improving the effect of driving training and the students' skill level.

[0010] The above description is only an overview of the technical solution of the present application. In order to more clearly understand the technical means of the present application, it can be implemented in accordance with the contents of the specification. In order to make the above and other purposes, features and advantages of the present application more obvious and easy to understand, the specific implementation methods of the present application are listed below. BRIEF DESCRIPTION OF THE DRAWINGS

[0011] Figure 1 A flowchart of a driving skill training method combined with virtual reality is provided for an embodiment of the present application;

[0012] Figure 2 A structural schematic diagram of a driving skill training system combined with virtual reality is provided for an embodiment of the present application.

[0013] Explanation of the reference numerals: driving scenario generating module 11, simulation data set acquiring module 12, scoring model building module 13, scoring result output module 14, training report generating module 15. DETAILED DESCRIPTION

[0014] The overall idea of ​​the technical solution provided by this application is as follows:

[0015] The embodiments of the present application provide a driving skills training method and system combined with virtual reality. By combining virtual reality (VR) technology, a virtual reality driving simulator is used to generate multiple driving scenarios, and trainees perform driving simulations and collect data in these scenarios. By constructing a driving operation scoring model, each set of simulation data is analyzed, the driving scoring results are output, and a personalized driving skills training report is generated based on these scoring results. This solution realizes digitized and personalized evaluation and feedback, and can provide trainees with targeted training and improvement suggestions, thereby improving the efficiency and effectiveness of driving skills training.

[0016] After introducing the basic principles of the present application, various non-limiting implementation methods of the present application will be specifically described below in conjunction with the drawings in the specification.

[0017] Embodiment 1, as Figure 1 As shown, the embodiment of the present application provides a driving skill training method combined with virtual reality, the method comprising:

[0018] Step S100: connecting a virtual reality driving simulator, and generating a plurality of driving scenes according to the virtual reality driving simulator, wherein the virtual reality driving simulator includes a VR wearable device.

[0019] Specifically, a virtual reality driving simulator is a system that uses virtual reality (VR) technology to create a virtual driving environment. By simulating real driving scenarios, it provides users with a training experience similar to actual driving. Driving simulators generally include hardware devices (such as consoles, steering wheels, pedals, seats, etc.) and software platforms (such as virtual environment generation and interaction systems). VR wearable devices refer to hardware devices that allow users to immerse themselves in a virtual environment, usually including virtual reality helmets (such as Oculus Rift, HTC Vive, Meta Quest, etc.) and supporting sensors. After wearing this device, users can perceive and interact with the content in the virtual world through vision, hearing, and even touch to achieve an immersive effect. Driving scenarios refer to various virtual driving environments presented in driving simulators. It includes multiple factors such as road conditions, traffic conditions, and weather changes. For example, various driving situations such as urban roads, mountain roads, and complex intersections can be simulated.

[0020] First, the driving simulator is connected to the hardware equipment through the software platform to form a complete training system. The simulator can present various driving environments in real time by connecting to the computer system or control unit. The hardware equipment includes the steering wheel, brake pedal, accelerator pedal, seat, etc., which simulates the real driving operation feeling.

[0021] Once the virtual reality system is established, the system software will use 3D modeling and rendering technology to generate different driving scenarios. These scenarios can include urban roads, mountain roads, highways, severe weather conditions, etc. to meet different training needs. For example, in urban road scenarios, factors such as traffic lights, pedestrians, and other vehicles are simulated; in mountain road scenarios, terrain features such as sharp turns, uphill and downhill are simulated.

[0022] VR wearable devices (such as Oculus Rift, HTC Vive, etc.) connect students to the virtual environment through virtual reality helmets. After wearing the helmet, students can perceive the 360-degree virtual scene through head movement and interact with the virtual environment through handheld controllers. For example, when students turn their heads, the sensors in the helmet will update the display in real time to present the driver's perspective, thereby simulating the real driving experience.

[0023] In the virtual reality system, driving scenarios include not only different geographical environments, but also different driving difficulties. For example, students can be trained in basic driving skills in ordinary urban road scenarios, or more complex driving environments can be simulated at night or in rainy days to increase the challenge of training.

[0024] By connecting to a virtual reality driving simulator and generating multiple driving scenarios, trainees can practice repeatedly in a virtual environment and master the coping skills for different driving situations. Since the system can provide a variety of complex driving environments, trainees can conduct all-weather training without being restricted by time and location. This immersive training greatly improves trainees' understanding and coping capabilities of driving techniques and reduces the safety risks encountered in actual road training.

[0025] Step S200: The training user performs driving simulation in the multiple driving scenarios according to the VR wearable device to obtain multiple groups of driving simulation data sets, wherein each group of driving simulation data sets corresponds to one driving scenario.

[0026] Specifically, training users refer to trainees who participate in driving skills training, usually those who undergo driving simulation and skills training through virtual reality driving simulators. Training users can be novice drivers or those who already have certain driving experience. According to different training needs, the system will provide personalized driving scenarios. Driving simulation data sets refer to all the operation data of trainees in specific scenarios recorded by the system during the driving simulation process. Each data set usually contains the behavior of trainees during driving, such as acceleration, braking, steering, reaction speed, vehicle speed, etc. These data are an important basis for evaluating trainees' driving skills.

[0027] The training users first put on the VR helmet and controller and enter the virtual reality system through the VR wearable device. After wearing the device, the user's vision is completely immersed in the virtual driving scene, as if they were in the actual driving environment. For example, the user can see the streets, traffic lights, pedestrians, etc. in the city road scene, as if they were in actual driving.

[0028] The system customizes different driving scenarios for each student. For example, the first scenario is a sunny city road, simulating the student's performance when driving in a busy urban area; the second scenario is a rainy highway, requiring the student to deal with special conditions such as slippery roads, rainy and foggy weather; the third scenario is a mountain road at night, simulating the difficulty of night driving. Students choose scenarios for simulated driving according to their own training plans.

[0029] In each driving scenario, the system will record the trainee's driving behavior in real time, including but not limited to acceleration, braking, steering, speed changes, reaction speed, concentration, etc. For example, in the urban road scenario, the system will record whether the trainee can brake in time when encountering a red light, whether he obeys traffic rules, whether he frequently brakes or accelerates, etc. In the rainy highway scenario, the system will record the trainee's speed control, whether he maintains a safe distance, and driving stability on slippery and wet roads.

[0030] Each set of driving simulation data sets corresponds to a specific driving scenario. For example, in an urban road scenario, the system will record all driving behavior data to form a set of data sets; in a rainy highway scenario, another set of data sets will record the trainee's operating performance in this scenario. In this way, the system can generate independent data sets for each scenario, which is convenient for subsequent analysis and scoring.

[0031] Through this step, the system can capture the trainee's performance in different driving environments in real time and comprehensively, providing data support for subsequent skill assessment. The simulated data set for each scenario not only helps to analyze the trainee's driving ability under specific conditions, but also provides a basis for the formulation of personalized training programs. For example, if the trainee performs poorly in rainy weather scenarios, the system can arrange more rainy weather driving training in a targeted manner based on data feedback. Through the accumulation of multiple scenarios and multiple data sets, trainees can gradually improve their ability to cope with various driving environments and ultimately achieve comprehensive driving skills improvement.

[0032] Step S300: constructing a driving operation scoring model, wherein the driving operation scoring model is connected to the virtual reality driving simulator.

[0033] Specifically, the driving operation scoring model is an algorithmic model used to evaluate the driver's operating performance during simulated driving. The model analyzes the trainee's various operating behaviors (such as acceleration, braking, steering, etc.) in the driving simulation and gives a comprehensive score to reflect the trainee's driving skill level and operating safety. The scoring model is usually based on specific driving behavior characteristics, such as reaction time, driving smoothness, safety and other indicators.

[0034] When establishing a driving operation scoring model, we first need to define the scoring criteria and evaluation indicators of the model. For example, the model scores students based on the following behaviors during driving: Braking characteristics: whether they brake promptly and smoothly when necessary, and whether the braking force is appropriate; Steering characteristics: whether they maintain a smooth steering and avoid sharp turns; Speed ​​control: whether they adjust the speed reasonably according to the changes in road conditions to avoid driving too fast or too slow; Reaction characteristics: whether the student's reaction speed is timely and effective when encountering emergencies (such as changes in traffic signals, obstacles, etc.).

[0035] To achieve these scoring criteria, the model can use machine learning or rule engines to automatically calculate a comprehensive score based on the student's operating data. For example, the model can analyze the student's behavioral data in the simulation scenario, identify the student's driving habits, and assess the quality of the driving operation based on these habits.

[0036] Specifically, to construct a driving operation scoring model, first, define the scoring criteria and evaluation indicators, and clarify the driving behavior characteristics that need to be evaluated. Each characteristic should be quantified according to actual driving requirements to form a computable parameter. Secondly, collect the student's operation data in the virtual reality driving simulator, including acceleration, braking, steering, vehicle speed, reaction time and other information. Then, select a suitable algorithm to build a scoring model. A rule-based model can be used to judge the quality of the student's operation based on a preset threshold; or a machine learning algorithm, such as a decision tree, support vector machine (SVM), etc., can be used to train the model through historical data so that it can automatically identify and score the student's driving behavior. Next, design a data transmission mechanism to transmit the real-time data in the simulator to the scoring model for processing. Finally, verify and optimize the scoring model to ensure its accuracy and stability. By continuously debugging the model parameters, the rationality and personalization of the scoring results can be improved, and finally a comprehensive evaluation of the student's driving skills can be achieved.

[0037] The driving operation scoring model needs to establish a data connection with the virtual reality driving simulator. This is usually achieved through an API interface or data transmission protocol. The simulator will send the student's operation data (such as acceleration, braking, steering angle, vehicle speed, etc.) to the scoring model in real time. After receiving this data, the scoring model analyzes it according to pre-set rules and algorithms, and finally outputs the student's operation score in each driving scenario.

[0038] By building a driving operation scoring model and connecting it to a virtual reality driving simulator, the system can provide an objective and quantitative driving evaluation to help students understand their own driving strengths and weaknesses and adjust their training plans based on the scoring results.

[0039] Step S400: inputting the plurality of driving simulation data sets into a driving operation scoring model, analyzing each driving simulation data set according to the driving operation scoring model, and outputting a plurality of driving operation scoring results.

[0040] Specifically, the driving operation score is the score output by the model after analyzing each set of simulated data sets. These results reflect the student's driving skill level in a specific scenario. For example, the score will show how the student performs in braking, steering, reaction speed, etc.

[0041] During the training process, each driving behavior of the trainee (such as braking, acceleration, steering, etc.) is recorded to form independent data sets. The system will input these data sets into the scoring model according to the scenario.

[0042] After receiving the data, the driving operation scoring model will analyze it based on preset rules or machine learning algorithms. The model will first extract features from the data and identify important driving behavior features (such as braking features, steering angles, speed changes, etc.). Then, each feature will be scored according to the set evaluation criteria, and finally a comprehensive operation score will be obtained.

[0043] Each data set corresponds to a specific driving scenario. After evaluating each data set, the scoring model outputs a driving operation score. For example, when simulating urban road driving, the model will output the student's scores in terms of braking, acceleration, reaction speed, etc.; in the rainy highway scenario, the model will evaluate the student's ability to cope with slippery roads and the level of speed control. Each scoring result reflects the student's skill level in a specific driving scenario.

[0044] Through this step, the system can achieve a comprehensive analysis of the student's driving behavior and automatically and accurately evaluate the student's driving ability in different scenarios. The scoring results of each driving scenario can reflect in detail the student's performance in a specific driving situation, such as whether the brakes are timely, whether the speed is appropriate, whether the reaction is quick, etc. This can not only provide students with accurate skill feedback, but also help coaches develop personalized training plans, train and improve students' weak links, and thus improve students' comprehensive driving ability.

[0045] Step S500: generating a driving skill training report according to the multiple driving operation scoring results.

[0046] Specifically, the driving skills training report is a comprehensive document that shows the performance evaluation results of the trainee in various aspects of the training process. The report not only contains the trainee's score in each scenario, but also can include the trainee's overall performance trend, improvement direction and future training suggestions. The report is usually used to help trainees and coaches understand the progress of the trainee and make reasonable plans for subsequent training.

[0047] First, the system collects the students' rating results in various driving scenarios from the previous training sessions. These rating results include ratings in multiple dimensions (for example, driving stability, reaction speed, operational fluency, etc.). These results are digital ratings obtained by analyzing the students' performance in different scenarios through the driving operation rating model.

[0048] The system will summarize and analyze the collected scoring results. Here, evaluation indicators (such as driving stability, reaction time, safety, etc.) will be summarized into specific performances in each scenario, and even summarized through certain data processing methods (such as weighted average, maximum value, etc.).

[0049] The key part of generating reports is the comprehensive performance analysis of the scoring results of multiple scenarios. For example, the system will calculate the average score of the students in all scenarios to find out the strengths and weaknesses. Based on the student's performance, the system will automatically generate training suggestions based on the data analysis results. Finally, the system will generate a driving skills training report with all the analysis results in a clear and intuitive form. The report is usually presented in the form of charts, text and tables, which is easy for students and coaches to understand. The charts can include the student's score curve in various dimensions, comparison charts of various scenarios, etc.

[0050] Through this process, trainees can clearly understand their performance and progress in the training process, discover their strengths and weaknesses in a timely manner, and then develop targeted improvement plans. At the same time, coaches or training institutions can also adjust the training content based on the content of the report and provide more personalized training programs. The report is not only a feedback tool for trainees, but also an important means to improve the overall training quality, which helps trainees to continuously optimize their driving skills.

[0051] Furthermore, each group of driving simulation data sets is analyzed according to the driving operation scoring model, and the method includes: the driving operation scoring model performs feature extraction on each group of driving simulation data sets to obtain low-dimensional features, and the low-dimensional features include braking features, speed features, steering features, and reaction features; high-dimensional feature convolution is performed according to the low-dimensional features to output high-dimensional features, and the high-dimensional features include driving stability, driving smoothness, reaction rate, and driving safety; the driving operation scoring model performs analysis according to the high-dimensional features to obtain the driving operation scoring results corresponding to each group of driving simulation data sets.

[0052] Specifically, low-dimensional features are basic, concise descriptive features extracted from the original driving data, and are usually single indicators that directly reflect the student's driving operation. For example, braking features include braking force and timing; speed features refer to the change in vehicle speed; steering features describe the steering angle and the smoothness of steering; and reaction features involve the student's reaction speed and decision-making to emergencies. These low-dimensional features are the basis of high-dimensional features and provide a preliminary basis for subsequent analysis. High-dimensional features are more complex and abstract features obtained by combining, calculating or convolving low-dimensional features. They can comprehensively reflect the overall performance of the student during driving. For example, driving stability can be comprehensively evaluated by multiple low-dimensional features such as braking smoothness and vehicle speed control stability; driving fluency involves the consistency and fluency of operations during driving. Feature convolution is the process of combining and calculating low-dimensional features through mathematical methods (such as convolutional neural networks or other algorithms) to obtain high-dimensional features. The convolution operation is used in this step to extract deeper driving behavior patterns, making the analysis results obtained from low-dimensional features more comprehensive and accurate. Driving stability refers to whether the trainee can control the vehicle smoothly during driving and avoid sudden braking or sharp turns. Driving fluency measures the consistency and smoothness of operations during driving, especially the connection between steering and acceleration and braking. Reaction rate refers to the speed and accuracy of the trainee's response to unexpected situations. Driving safety refers to the evaluation of whether the trainee follows safe driving rules, such as complying with speed limits, keeping a reasonable distance from the vehicle in front, and responding to traffic signals in a timely manner.

[0053] During the driving simulation, the system will collect various operational data of the trainees in real time. First, these data will be decomposed into a series of low-dimensional features, namely simple operational indicators, such as braking features, speed features, steering features, and reaction features. For example, braking features include the time delay and strength of braking reaction; steering features include steering angle and steering smoothness; reaction features describe the trainees' response speed when facing traffic light changes or emergency situations. All these low-dimensional features will provide basic data for the calculation of high-dimensional features.

[0054] Next, the scoring model performs high-dimensional feature convolution on the extracted low-dimensional features. Through the convolution operation, the system combines different low-dimensional features to generate comprehensive, high-level features. For example, driving stability can be obtained by combining the smoothness of braking with the stability of vehicle speed control; driving smoothness is calculated from multiple low-dimensional features such as the coordination between acceleration and braking and the smoothness of steering. This convolution process enables the model to integrate multiple data and analyze the student's overall driving performance, rather than just a single operating indicator.

[0055] After convolution processing, the model will generate a series of high-dimensional features, such as driving stability, driving fluency, reaction rate, and driving safety. These high-dimensional features can more comprehensively and accurately reflect the student's driving ability. For example, if the student reacts smoothly when braking and controls the speed steadily, the model will output a high driving stability score; if the student turns smoothly and reacts quickly when changing lanes, the fluency score will be high.

[0056] Finally, the scoring model comprehensively analyzes each set of driving simulation data sets based on these high-dimensional features and outputs the corresponding driving operation scoring results. Each set of scoring results will reflect the student's overall performance in that driving scenario. For example, in the city driving simulation, if the student brakes in time and steers smoothly, the score result will be high; if the student reacts slowly and controls the speed improperly in the highway scenario, the score result will be low.

[0057] Through this feature extraction and convolution analysis process, the scoring model can more comprehensively and accurately evaluate the student's driving ability. Compared with relying solely on low-dimensional features, the combination of high-dimensional features can provide more detailed feedback to help students understand their specific performance during driving.

[0058] Furthermore, after outputting a plurality of driving operation scoring results, the method further includes: setting a plurality of driving scoring levels for each driving scenario; performing scoring level matching in each driving scenario according to the plurality of driving operation scoring results to obtain a matching driving scoring level in each driving scenario; and integrating the plurality of matching driving scoring levels in the plurality of driving scenarios to generate a comprehensive driving scoring level.

[0059] Specifically, the driving rating level is a categorized evaluation of the student's operating performance in a specific driving scenario. The student's driving level is usually divided into different levels according to the rating results. The rating level can be numerical (such as 1 to 100 points) or graded (such as excellent, good, medium, poor, etc.). Different rating levels represent the student's skill level in the driving scenario. Rating level matching is to compare the student's driving operation rating results with the preset rating standards to determine the student's level in a specific scenario. The rating level matching of each scenario will determine the student's specific score in the scenario based on the student's performance. The comprehensive driving rating level is a comprehensive evaluation result obtained by weighting or averaging the rating results in multiple driving scenarios, reflecting the student's overall performance in all simulated scenarios. This comprehensive rating can provide students with a comprehensive skill assessment, so that students can understand their driving ability in different situations.

[0060] First, set specific scoring standards for each driving scenario. For example, the scoring levels for urban road driving scenarios can be set as: excellent, good, medium, pass, and fail. This setting divides the student's performance into different levels based on the operation score results calculated by the scoring model. Similar scoring levels can also be set for other scenarios, such as highways, mountain roads, etc.

[0061] Compare the student's score results in each driving scenario with the set score level to determine the student's score level in each scenario. For example, in the city driving scenario, if the student's score is 85 points, according to the score level standard, his level is "good"; if in the highway scenario, the score is 75 points, it is "medium". The score level of each scenario corresponds to the student's performance in that specific scenario.

[0062] After the student completes the simulation training of multiple scenarios, the system will integrate the ratings in each scenario. Usually, this integration process can be done by weighted average to combine the ratings of all scenarios. For example, the ratings in different scenarios such as city driving, mountain driving, and highway driving can be assigned weights according to their importance. For example, highway driving is considered more challenging and is therefore given a higher weight. Finally, the system will output a comprehensive driving rating that reflects the student's overall performance in all scenarios. For example, assuming that the student performs well in both city driving and mountain driving, but performs moderately in highway driving, then the comprehensive rating will be "good" or "moderate", and the specific rating is calculated based on the weight of each scenario.

[0063] This process enables the scoring system to comprehensively evaluate the student's driving ability based on performance in different driving scenarios. By converting the scoring results of each scenario into a specific grade and weighting them according to importance, a comprehensive comprehensive scoring grade is finally generated. Students can not only see their specific performance in each scenario, but also get a comprehensive and objective skill evaluation. This comprehensive score can help students fully understand their strengths and weaknesses, so as to improve and enhance them in a targeted manner. At the same time, coaches or training institutions can develop more targeted training plans based on the students' comprehensive scores to further improve the students' driving skills.

[0064] Furthermore, after outputting a plurality of driving operation scoring results, the method further includes: feeding back the plurality of driving operation scoring results to the virtual reality driving simulator to generate a plurality of driving training scenarios, and the training user performs training based on the plurality of driving training scenarios; the virtual reality driving simulator compares the plurality of driving operation scoring results with a pre-stored driving operation scoring threshold, and screens N driving scenarios that are less than the driving operation scoring threshold, wherein N is a positive integer greater than or equal to 1; and identifying the N driving scenarios as a plurality of driving training scenarios to be trained by the current training user.

[0065] Specifically, the driving operation scoring threshold is a pre-set score or standard that represents the minimum or expected driving performance that the trainee should achieve in a certain scenario. The scoring threshold can be set according to the training purpose, scenario difficulty or student level. For example, the scoring threshold of a certain scenario requires that the student's driving stability score is more than 80 points in order to pass the training of the scenario. The N driving scenarios refer to a set of driving simulation scenarios screened according to the scoring threshold, where N is a positive integer greater than or equal to 1, indicating the number of scenarios in which the student performs poorly in multiple driving scenarios and requires further training. The screened scenarios will serve as the trainee's training targets. The multiple driving training scenarios to be trained are specific scenarios extracted from all driving scenarios through screening operations and that the trainee needs to train again. Typically, these scenarios are scenarios in which the trainee performed poorly in the first simulation training and scored below the threshold.

[0066] The performance of the trainees in multiple driving simulation scenarios has been evaluated through the scoring model. These driving operation scoring results include evaluation data of various driving skill indicators. These scoring results will be fed back to the virtual reality driving simulator and used as a basis for decision-making on the next training content. The virtual reality driving simulator will generate a series of new driving training scenarios based on the scoring results to strengthen the training of the trainees' shortcomings.

[0067] The virtual reality driving simulator compares the driving operation scoring results in each scene with the pre-stored driving operation scoring threshold. Each driving scene has a set scoring standard (i.e. threshold). When the student's score is lower than the threshold, the simulator will consider that the scene needs more training. For example, if the scoring threshold of a scene is set to 70 points and the student scores 65 points, the scene will be marked as a scene that needs retraining.

[0068] Based on the comparison results, the virtual reality driving simulator will select multiple scenarios with scores below the scoring threshold, the number of which is N, where N is a positive integer (greater than or equal to 1). These selected scenarios are the scenarios that the trainees currently need to focus on training. The trainees will conduct more in-depth training in these scenarios to improve their weak links.

[0069] The selected driving scenarios will be marked as "pending training" scenarios, and students will receive intensive training in these specific scenarios. For example, students are required to practice in more complex urban roads or highways to improve their braking response or steering control accuracy. The student's training system will be adaptively adjusted according to these pending training scenarios to ensure that the training content is highly targeted.

[0070] This step realizes a personalized training plan, which can adjust the training content in real time according to the students' performance in different scenarios. Through the feedback system, students can focus on the scenes where they perform poorly and make targeted improvements.

[0071] Furthermore, a plurality of driving scenarios are generated according to the virtual reality driving simulator, and the method includes: constructing a driving scenario database; inputting the identity information and training project information of the training user; authenticating the identity information of the training user, and if the identity authentication is passed, filtering the project scenarios in the driving scenario database according to the training project information, and outputting a filtered scenario set; and generating a plurality of driving scenarios corresponding to the training user according to the filtered scenario set.

[0072] Specifically, the driving scenario database is a database system that stores various driving simulation scenario data. It contains a variety of driving scenarios and environmental settings, such as urban roads, highways, mountain roads and other common or special driving scenarios. Each scenario contains specific information related to it, such as scenario description, environmental conditions, traffic conditions, road structure, etc. The identity information of the training user refers to the personal identity information of the trainee in the training system, including name, age, driver's license information, trainee number, etc. This information is used to uniquely identify the trainee and track and record it during the training process. Training project information refers to the training content, course schedule, goals and other information related to the training user. For example, the trainee participates in "basic driving skills training" or "advanced driving skills training". Different training projects involve different driving scenarios and simulation training requirements. Project scenario screening is the process of selecting a set of driving scenarios related to the project from the driving scenario database based on the training project information. The screened scenario set refers to a set of driving simulation scenarios screened from the database according to the training project requirements. These scenarios are customized to meet specific training goals, and the trainees will be trained in these scenarios.

[0073] First, build a driving scenario database and store different driving scenarios and environmental conditions in it. This database contains a rich variety of scenario types, such as urban roads, highways, mountain roads, etc., to meet different training needs and situations. Specifically, first define the structure of the database, including information such as scenario types, environmental conditions, traffic conditions, road characteristics, etc. Then, collect and organize specific data for different driving scenarios, such as specific descriptions of each scenario such as urban roads, highways, mountain roads, traffic signs, weather conditions, speed limits, etc. Next, enter this data into the database, which can use a relational database (such as MySQL) or a non-relational database (such as MongoDB). In addition, the scenario needs to be associated with other data (such as user preferences, driving skill levels, etc.) to ensure that the system can filter and generate suitable training scenarios according to different needs.

[0074] Then, the training user's identity information and training project information are entered into the system, such as the trainee's name, driver's license level, training course content, etc. This information not only helps the system distinguish different trainees, but also helps select appropriate scenarios based on the trainee's training goals. For example, for beginners, the system will give priority to simple urban driving scenarios, while for trainees with certain driving experience, it will choose more challenging scenarios, such as mountain roads or complex traffic environments.

[0075] Next, the system will authenticate the identity information of the trainee to ensure that each trainee can obtain the correct personalized training content. If the identity authentication is passed (for example, the trainee successfully logs into the system and verifies his identity), the system will screen the project scenarios based on the trainee's training program information (such as course schedule or skill requirements). For example, if the trainee is participating in "Basic Driving Skills Training", the system will screen out simple driving scenarios related to this project, such as flat city roads or low-speed loop scenarios. If the student's project is "Advanced Driving Skills Training", the system will select more complex scenarios, such as night driving, driving in rainy days, or driving on mountain roads.

[0076] Finally, based on the selected set of scenarios, the system generates a series of driving scenarios that meet the needs of the training users. Each scenario is customized according to the trainee's training program to ensure the pertinence and effectiveness of the training. For example, if the trainee needs to practice highway driving, the system will generate a set of highway-related simulation training scenarios, including different traffic flows, weather conditions, traffic signs and other elements.

[0077] Through this process, the system can provide each student with personalized and targeted driving training. Students can get the most suitable training scenario according to their needs and training goals, which not only improves the efficiency of training, but also ensures that students can improve their driving skills in the shortest possible time.

[0078] Furthermore, after outputting the filtered scene set, the method also includes: wherein each driving scene in the driving scene database carries identification information, and the identification information is an indicator for measuring the difficulty of driving scene operation; obtaining an identification information set corresponding to the filtered scene set; identifying the uniformity of operation difficulty distribution for the identification information set, optimizing the multiple driving scenes with a preset uniformity of operation difficulty distribution as a goal, and outputting the optimized multiple driving scenes.

[0079] Specifically, in the driving scenario database, each driving scenario will have an identification information indicating the operational difficulty of the scenario. This is usually a numerical value or level used to measure the driving difficulty of the scenario. The operational difficulty can be set according to a variety of factors, such as traffic density, road complexity, weather conditions, etc. For example, the driving difficulty on highways is lower, while the difficulty on mountain roads or complex intersections in cities is higher. Identification of the uniformity of the distribution of operational difficulty refers to identifying the distribution of their operational difficulty in multiple screened driving scenarios and evaluating whether it is uniform. If the difficulty distribution of a set of scenarios is more concentrated, the training effect is not ideal and the improvement of the trainees' abilities is limited. The optimized multiple driving scenarios are achieved by adjusting and re-screening the distribution of operational difficulty so that trainees can experience more balanced challenges. For example, if there are too many scenes with lower difficulty in a training project, the optimization process will introduce more scenes with higher difficulty to ensure that the trainees' skills are trained in a balanced manner.

[0080] In the previous step, the system has screened out multiple scenarios from the driving scenario database based on the needs of the trainees. These scenarios all carry an operation difficulty identification information, that is, the preset difficulty value of each scenario. The system will extract these identification information sets from the screened scenarios to obtain the difficulty values ​​of all scenarios. For example, for the screened scenarios including highways, urban roads, and mountain roads, the system will extract the difficulty values ​​of these scenarios, which are 3, 5, and 8 respectively.

[0081] The system will identify the uniformity of the operation difficulty distribution based on the extracted difficulty value set. The goal of this identification process is to check whether the difficulty distribution of the current screening scene is uniform. For example, if the difficulty values ​​of most scenes are concentrated in a lower range (such as 2 to 4), and there are not enough high-difficulty scenes, then the difficulty distribution is uneven and needs to be adjusted. The system uses statistical algorithms (such as uniformity calculation, variance analysis, etc.) to determine the uniformity of the current difficulty distribution.

[0082] Based on the results of the uniformity of the operation difficulty distribution, the system will optimize the current screening scene. The purpose of the optimization is to make the difficulty of the scene set more balanced. For example, if the system finds that the difficulty of the current training scene is too simple, the optimization process will add some more challenging scenes, or adjust the scene order, so that students can gradually improve their skills and eventually achieve a uniform difficulty training experience.

[0083] After the optimization process is completed, the system will generate a new set of optimized scenes with a more reasonable difficulty distribution. For example, the system extracts several sets of medium-difficulty scenes from low-difficulty urban roads and highway scenes, and then adds some more challenging mountain roads, complex intersections or special weather conditions scenes to ensure that students gradually improve from simple to complex and from easy to difficult.

[0084] Through this process, trainees can receive more balanced and gradually improved training content, avoiding situations where the training difficulty is too high or too low. By optimizing the difficulty distribution, the system can provide trainees with more appropriate training challenges, helping them gradually master more difficult driving skills in the process of continuous improvement, and improving training results.

[0085] Furthermore, the training user performs driving simulation in the multiple driving scenarios according to the VR wearable device, and the method also includes: obtaining the cumulative simulation time of the training user in the multiple driving scenarios; if the cumulative simulation time is greater than the preset cumulative simulation time, generating a first reminder message, and the virtual reality driving simulator reminds the training user to suspend training according to the first reminder message.

[0086] Specifically, the cumulative simulation time refers to the total time that the training users conduct driving simulation training in the virtual reality driving simulator. It is a statistics of the user's cumulative operation time in multiple driving scenarios, which is used to measure the trainee's training time. The preset cumulative simulation time is a maximum training time limit set by the system to prevent users from conducting simulation training for too long and avoiding fatigue that affects the training effect. The duration can be set according to the trainee's training needs, physical condition, and the optimal training time recommended by scientific research. The first reminder message refers to the reminder message automatically generated by the system when the trainee's cumulative simulation time exceeds the preset time. The purpose of this reminder message is to remind trainees to rest or pause training to avoid excessive fatigue and ensure training results and physical health.

[0087] After the trainee puts on the VR wearable device and enters the virtual driving simulation environment, the system will track and record the trainee's simulated driving time in real time. The system can accurately calculate the trainee's training time in each driving scenario, and accumulate these times to form the trainee's cumulative simulation time.

[0088] The system will set a preset cumulative simulation time based on the trainee's training needs and health recommendations. For example, if the preset cumulative simulation time is 2 hours, then when the total training time of the trainee in the virtual reality system exceeds 2 hours, the system will trigger a reminder mechanism. The preset time can also be flexibly adjusted according to different scenarios and the trainee's training conditions to avoid physical fatigue or decreased concentration of the trainee due to overtraining.

[0089] Once the system detects that the student's cumulative simulation time exceeds the preset value, the system will automatically generate a first reminder message to prompt the student to rest or pause the training. The system will convey this information to the student through the interface of the virtual reality driving simulator or the voice reminder function. After the student sees the prompt message, the system can also pause the training process or forcefully terminate the current driving scene as needed. This mechanism ensures that the student will not incur unnecessary health risks due to overtraining.

[0090] Through this step, the system can effectively manage the training time of students, avoid the accumulation of fatigue caused by long-term training, and thus improve the efficiency and effect of training. Controlling the training time not only helps to improve the concentration of students, but also protects their physical health and reduces the potential risks caused by fatigue driving.

[0091] In summary, the driving skill training method combined with virtual reality provided in the embodiments of the present application has the following technical effects:

[0092] 1. By combining a virtual reality driving simulator with a driving operation scoring model, an immersive training environment is provided, and accurate data-based evaluation of trainees’ performance is achieved. This solution can objectively reflect trainees’ operational capabilities in different driving scenarios, providing a reliable basis for personalized training, thereby improving training effectiveness and trainees’ skill levels.

[0093] 2. By building a driving scenario database and screening appropriate training scenarios based on the trainees’ identity information, the training content can be customized. This solution makes the training more in line with the trainees’ needs, improves the training efficiency, and ensures that the trainees are trained in scenarios that suit their abilities, thereby better improving their driving skills.

[0094] 3. By monitoring the cumulative simulation time of students and providing reminders, students can avoid fatigue caused by long-term training and improve the scientificity and safety of training. This mechanism effectively manages the training time of students, ensures that students can maintain the best condition for driving training, and thus optimizes the training effect.

[0095] Embodiment 2, based on the same inventive concept as the driving skill training method combined with virtual reality in the above embodiment, Figure 2 As shown, the embodiment of the present application provides a driving skill training system combined with virtual reality, the system comprising:

[0096] A driving scenario generation module 11 is used to connect to a virtual reality driving simulator and generate multiple driving scenarios according to the virtual reality driving simulator, wherein the virtual reality driving simulator includes a VR wearable device; a simulation data set acquisition module 12 is used to train users to perform driving simulations under the multiple driving scenarios according to the VR wearable device, and obtain multiple groups of driving simulation data sets, wherein each group of driving simulation data sets corresponds to one driving scenario; a scoring model construction module 13 is used to construct a driving operation scoring model, wherein the driving operation scoring model is connected to the virtual reality driving simulator; a scoring result output module 14 is used to input the multiple groups of driving simulation data sets into the driving operation scoring model, analyze each group of driving simulation data sets according to the driving operation scoring model, and output multiple driving operation scoring results; a training report generation module 15 is used to generate a driving skill training report according to the multiple driving operation scoring results.

[0097] Furthermore, the scoring result output module 14 is also used to perform the following steps: the driving operation scoring model performs feature extraction on each group of driving simulation data sets to obtain low-dimensional features, and the low-dimensional features include braking features, speed features, steering features, and reaction features; high-dimensional feature convolution is performed based on the low-dimensional features to output high-dimensional features, and the high-dimensional features include driving stability, driving smoothness, reaction rate, and driving safety; the driving operation scoring model performs analysis based on the high-dimensional features to obtain the driving operation scoring results corresponding to each group of driving simulation data sets.

[0098] Furthermore, the scoring result output module 14 is also used to perform the following steps: setting multiple driving scoring levels for each driving scenario; matching the scoring levels in each driving scenario according to the multiple driving operation scoring results to obtain a matching driving scoring level in each driving scenario; integrating the multiple matching driving scoring levels in the multiple driving scenarios to generate a comprehensive driving scoring level.

[0099] Furthermore, the scoring result output module 14 is also used to perform the following steps: feeding back the multiple driving operation scoring results to the virtual reality driving simulator to generate multiple driving training scenarios, and the training user is trained based on the multiple driving training scenarios; the virtual reality driving simulator compares the multiple driving operation scoring results with a pre-stored driving operation scoring threshold, and screens N driving scenarios that are less than the driving operation scoring threshold, where N is a positive integer greater than or equal to 1; and identifying the N driving scenarios as multiple driving training scenarios to be trained by the current training user.

[0100] Furthermore, the driving scenario generation module 11 is also used to perform the following steps: construct a driving scenario database; input the identity information and training project information of the training user; authenticate the identity information of the training user, and if the identity authentication is passed, perform project scenario screening in the driving scenario database according to the training project information, and output a screening scenario set; and generate multiple driving scenarios corresponding to the training user according to the screening scenario set.

[0101] Furthermore, the driving scenario generation module 11 is also used to perform the following steps: wherein each driving scenario in the driving scenario database carries identification information, and the identification information is an indicator for measuring the difficulty of driving scenario operation; obtaining an identification information set corresponding to the screening scenario set; identifying the uniformity of operation difficulty distribution for the identification information set, optimizing the multiple driving scenarios with a preset uniformity of operation difficulty distribution as a goal, and outputting multiple optimized driving scenarios.

[0102] Furthermore, the simulation data set acquisition module 12 is also used to perform the following steps: obtaining the cumulative simulation time of the training user in the multiple driving scenarios; if the cumulative simulation time is greater than the preset cumulative simulation time, generating a first reminder message, and the virtual reality driving simulator reminds the training user to suspend training according to the first reminder message.

[0103] Any step of the method described above can be stored as a computer instruction or program in an unlimited computer memory, and can be called and recognized by an unlimited computer processor to implement any method in the embodiments of the present application, without any unnecessary restrictions.

[0104] Furthermore, the first or second mentioned above not only represents an order relationship, but also represents a specific concept, and / or refers to the selection of multiple elements individually or in whole. Obviously, those skilled in the art can make various changes and modifications to the present application without departing from the scope of the present application. Thus, if these modifications and variations of the present application fall within the scope of the present application and its equivalent technologies, the present application intends to include these modifications and variations.

Claims

1. A driving skills training method combined with virtual reality, characterized in that: The method comprises: Connecting a virtual reality driving simulator, and generating a plurality of driving scenarios according to the virtual reality driving simulator, wherein the virtual reality driving simulator includes a VR wearable device; The training user performs driving simulation in the multiple driving scenarios according to the VR wearable device to obtain multiple sets of driving simulation data sets, wherein each set of driving simulation data sets corresponds to one driving scenario; Constructing a driving operation scoring model, wherein the driving operation scoring model is connected to the virtual reality driving simulator; Inputting the plurality of driving simulation data sets into a driving operation scoring model, analyzing each driving simulation data set according to the driving operation scoring model, and outputting a plurality of driving operation scoring results; A driving skill training report is generated according to the multiple driving operation scoring results.

2. The method according to claim 1, characterized in that Analyzing each set of driving simulation data sets according to the driving operation scoring model, the method includes: The driving operation scoring model extracts features from each set of driving simulation data sets to obtain low-dimensional features, wherein the low-dimensional features include braking features, speed features, steering features, and reaction features; Perform high-dimensional feature convolution according to the low-dimensional features to output high-dimensional features, wherein the high-dimensional features include driving stability, driving smoothness, reaction rate, and driving safety; The driving operation scoring model performs analysis based on the high-dimensional features to obtain a driving operation scoring result corresponding to each group of driving simulation data sets.

3. The method according to claim 1, characterized in that After outputting the multiple driving operation scoring results, the method further includes: Set multiple driving scoring levels for each driving scenario; Performing score matching in each driving scenario according to the multiple driving operation score results to obtain a matching driving score in each driving scenario; A plurality of matching driving score levels under the plurality of driving scenarios are integrated to generate a comprehensive driving score level.

4. The method according to claim 1, characterized in that After outputting the multiple driving operation scoring results, the method further includes: Feeding back the multiple driving operation scoring results to the virtual reality driving simulator to generate multiple driving training scenarios, and the training user performs training based on the multiple driving training scenarios; The virtual reality driving simulator compares the multiple driving operation scoring results with a pre-stored driving operation scoring threshold, and selects N driving scenarios that are less than the driving operation scoring threshold, where N is a positive integer greater than or equal to 1; The N driving scenarios are identified as a plurality of driving training scenarios to be trained for the current training user.

5. The method according to claim 1, characterized in that Generating multiple driving scenarios according to the virtual reality driving simulator, the method comprises: Build a driving scenario database; Enter the identity information and training project information of the training user; Authentication is performed on the identity information of the training user. If the identity authentication is passed, project scenes are screened in the driving scene database according to the training project information, and a screened scene set is output; A plurality of driving scenarios corresponding to the training user are generated according to the screening scenario set.

6. The method according to claim 5, characterized in that After outputting the collection of filtered scenes, the method further includes: Wherein, each driving scene in the driving scene database carries identification information, and the identification information is an indicator for measuring the difficulty of operating the driving scene; Obtaining a set of identification information corresponding to the set of screening scenarios; The uniformity of the distribution of the difficulty of operation is identified for the identification information set, the multiple driving scenarios are optimized with the preset uniformity of the distribution of the difficulty of operation as a target, and the multiple optimized driving scenarios are output.

7. The method according to claim 1, characterized in that The method further includes: training a user to perform driving simulation in the plurality of driving scenarios according to the VR wearable device: Obtaining the cumulative simulation time of the training user in the multiple driving scenarios; If the accumulated simulation time is longer than the preset accumulated simulation time, a first reminder message is generated, and the virtual reality driving simulator reminds the training user to suspend the training according to the first reminder message.

8. Driving skills training system combined with virtual reality, characterized in that: The system is used to perform the driving skill training method combined with virtual reality according to any one of claims 1 to 7, the system comprising: A driving scenario generation module, used to connect to a virtual reality driving simulator and generate multiple driving scenarios according to the virtual reality driving simulator, wherein the virtual reality driving simulator includes a VR wearable device; A simulation data set acquisition module, used to train a user to perform driving simulation according to the VR wearable device in the multiple driving scenarios, and acquire multiple sets of driving simulation data sets, wherein each set of driving simulation data sets corresponds to one driving scenario; A scoring model building module, used to build a driving operation scoring model, wherein the driving operation scoring model is connected to the virtual reality driving simulator; a scoring result output module, configured to input the plurality of driving simulation data sets into a driving operation scoring model, analyze each driving simulation data set according to the driving operation scoring model, and output a plurality of driving operation scoring results; The training report generating module is used to generate a driving skill training report according to the multiple driving operation scoring results.

Images