Student safety operation monitoring device for driving skill training

Abstract

The invention provides a student safety operation monitoring device for driving skill training, and relates to the technical field of driving training monitoring, and the device comprises an initialization module which is used for obtaining the training data of a student, and carrying out the initialization; the external information acquisition module is used for acquiring real-time environment data; the internal information acquisition module is used for acquiring internal data; the state extraction module is used for performing feature extraction on the internal information data set; and the monitoring and early warning module is used for calling the initialized intelligent monitoring auxiliary system to perform joint anomaly authentication based on the internal feature extraction result and the external environment data set, and reporting monitoring and early warning. The technical problem that in the prior art, whether operation of a student is abnormal or not is judged depending on a single rule, false alarm or missing alarm is likely to occur, and the accuracy of safe operation monitoring is insufficient can be solved, combined authentication is carried out by combining behaviors of the student and external environment data, the abnormal condition is accurately recognized, and the safety operation monitoring accuracy is improved. And the safety operation monitoring accuracy is improved.

Description

Technical Field

[0001] The present application relates to the technical field of driving training monitoring, and in particular to a student safety operation monitoring device used for driving skill training. Background Art

[0002] In driving training, the monitoring of students' safe operation is the core part to ensure that students can have the necessary driving skills and safety awareness in actual driving. Through monitoring and evaluation, students can correct incorrect driving behaviors, improve their safety awareness, and reduce the risk of accidents. Safety operation monitoring in driving training usually involves real-time recording, analysis and evaluation of students' operations, providing feedback to students and making necessary warnings to guide students to improve their driving habits. At present, the safety operation monitoring of existing driving training systems mainly relies on on-board sensors, cameras, GPS, radar and other equipment to collect students' operation data and external environment data from different dimensions, and determine whether students' operations are safe based on pre-set thresholds. However, these simple rules often cannot accurately judge whether students' operations are reasonable, which can easily lead to false alarms or omissions, resulting in inaccurate monitoring.

[0003] In summary, the prior art has a technical problem of relying on a single rule to determine whether a trainee's operation is abnormal, which is prone to false alarms or missed alarms, resulting in insufficient accuracy in safety operation monitoring. Summary of the invention

[0004] The purpose of this application is to provide a student safety operation monitoring device for driving skill training, so as to solve the technical problem in the prior art that it relies on a single rule to judge whether the student's operation is abnormal, which is prone to false alarms or missed alarms, resulting in insufficient accuracy of safety operation monitoring.

[0005] In view of the above problems, the present application provides a student safety operation monitoring device for driving skills training, wherein the student safety operation monitoring device for driving skills training includes: an initialization module, used to obtain the student's training data, and perform system attention initialization of the intelligent monitoring assistance system based on the training data, the training data including the student's historical training data, current training scene data, and current training subject data; an external information acquisition module, used to call an external vehicle-mounted camera to perform real-time environmental data acquisition, and establish an external environment data set; an internal information acquisition module, used to call a built-in sensor network to perform internal data acquisition, and establish an internal information data set; a state extraction module, used to extract features of the student state and the vehicle state from the internal information data set, and establish an internal feature extraction result; a monitoring and early warning module, used to call the initialized intelligent monitoring assistance system to perform joint abnormality authentication based on the internal feature extraction result and the external environment data set, and report a monitoring and early warning.

[0006] The technical solution provided in this application has at least the following technical effects or advantages:

[0007] The initialization module is used to obtain the trainee's training data, and the system attention initialization of the intelligent monitoring assistance system is performed based on the training data. The training data includes the trainee's historical training data, current training scene data, and current training subject data; the external information acquisition module is used to call the external vehicle-mounted camera to perform real-time environmental data acquisition and establish an external environment data set; the internal information acquisition module is used to call the built-in sensor network to perform internal data acquisition and establish an internal information data set; the state extraction module is used to extract the characteristics of the trainee's state and vehicle state from the internal information data set and establish an internal feature extraction result; the monitoring and early warning module is used to call the initialized intelligent monitoring assistance system to perform joint abnormal authentication based on the internal feature extraction result and the external environment data set, and report a monitoring and early warning. In other words, by integrating the trainee's historical training data, current training scene data, and current training subject data, collecting real-time environment and internal data, extracting the characteristics of the trainee and vehicle status, and combining the joint abnormal authentication to perform proofreading and verification in multiple dimensions, the accuracy of safe operation monitoring is improved, ensuring the trainee's safe operation, thereby improving the safety of road traffic. BRIEF DESCRIPTION OF THE DRAWINGS

[0008] In order to more clearly illustrate the technical solutions in the present application or the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings in the following description are only exemplary, and for ordinary technicians in this field, other drawings can be obtained based on the provided drawings without paying any creative work.

[0009] Figure 1 This is a schematic diagram of the structure of the student safety operation monitoring device used for driving skills training in this application;

[0010] Figure 2 This is a flow chart of the monitoring warning issued by the student safety operation monitoring device used for driving skill training in this application.

[0011] Explanation of the accompanying drawings: initialization module 11, external information collection module 12, internal information collection module 13, state extraction module 14, monitoring and early warning module 15. DETAILED DESCRIPTION

[0012] This application solves the technical problem in the prior art that the safety operation monitoring is not accurate enough because it relies on a single rule to judge whether the student's operation is abnormal, which is prone to false alarms or missed alarms. By integrating the student's historical training data, current training scenario data, and current training subject data, collecting real-time environment and internal data, extracting the characteristics of the student and vehicle status, and combining joint abnormality authentication to perform proofreading and verification in multiple dimensions, the accuracy of safety operation monitoring is improved, ensuring the student's safe operation, thereby improving the safety of road traffic.

[0013] Below, the technical solutions in the present application will be clearly and completely described with reference to the accompanying drawings. Obviously, the described embodiments are only part of the embodiments of the present application, rather than all of the embodiments of the present application. It should be understood that the present application is not limited to the example embodiments described herein. Based on the embodiments of the present application, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present application. It should also be noted that, for the convenience of description, only the parts related to the present application are shown in the accompanying drawings, rather than all of them.

[0014] For examples, please see the attached Figure 1 The present application provides a student safety operation monitoring device for driving skill training, wherein the student safety operation monitoring device for driving skill training comprises:

[0015] The initialization module 11 is used to obtain the trainee's training data and perform system attention initialization of the intelligent monitoring assistance system based on the training data. The training data includes the trainee's historical training data, current training scenario data, and current training subject data.

[0016] Specifically, the trainee's training data is collected from the trainee's personal file or training database, that is, various types of data accumulated by the trainee during the driving training process, including the trainee's historical training records, information on the current training subject, and data in the training scenario. The historical training data records the trainee's previous training process, passed subjects, training results, and the trainee's driving behavior patterns. The current training scenario data refers to the trainee's current driving environment and training scenario data, including weather, road conditions (such as urban roads, highways), and traffic flow. The current training subject data refers to the specific training content that the trainee is currently undergoing, including but not limited to Subject 1 (theoretical examination), Subject 2 (parking, reversing into the garage, etc.), and Subject 3 (road driving, changing lanes, etc.). Each subject has different training goals and requires different monitoring based on the trainee's performance.

[0017] The intelligent monitoring assistance system is a comprehensive auxiliary monitoring system that combines a variety of sensors, cameras, algorithm models and other tools to monitor the trainees' driving behavior in real time and provide feedback or warnings based on their performance. The intelligent monitoring assistance system is initialized based on the training data, that is, before the intelligent monitoring assistance system starts running, it is configured and set up based on the trainees' training data to ensure that the system can monitor according to specific needs (the specific situation of the trainees). According to the trainees' training data, set thresholds and configure monitoring targets (such as checking whether the trainees can park accurately, whether they can successfully complete reverse parking, etc.). After initialization, start the real-time monitoring task and collect and analyze data for the content of the current training subject. By integrating multi-source data, the intelligent monitoring assistance system accurately understands the trainees' training needs and status, and conducts targeted personalized monitoring based on the trainees' historical data and current scene information, making the monitoring results more accurate.

[0018] The external information collection module 12 is used to call an external vehicle-mounted camera to perform real-time environmental data collection and establish an external environmental data set.

[0019] Specifically, an external vehicle camera is a visual acquisition device installed on the outside of a car, usually used to record and analyze environmental information around the car, obtain real-time image and video data around the vehicle, and help perceive the external environment, such as traffic signs, other vehicles, pedestrians, road obstacles, etc. During driving training, external vehicle cameras continuously monitor the environment around the trainees, usually installed in the front and rear or left and right sides of the car, to obtain images and video streams around the vehicle.

[0020] The external vehicle camera continuously collects various data of the current environment, including images, videos, and key information extracted from the camera, such as road signs, pedestrian positions, vehicle positions, weather conditions, etc. Through the external vehicle camera collection, the external environment data set is established, including various types of information related to the training environment where the trainees are located, such as road information, traffic conditions, weather conditions, etc. Road information includes road type (city streets, highways, etc.), number of lanes, whether there are obstacles, whether there are turns, and whether there are intersections; traffic information includes the status of traffic lights (red, green, yellow), traffic signs (speed limit signs, stop signs, etc.), surrounding vehicles and pedestrians, etc.; weather conditions include whether it is raining, smog, daytime or nighttime, etc., all of which will affect driving safety; dynamic environment includes surrounding vehicle speeds, pedestrian movement, traffic flow, etc. Through real-time data collection from the external vehicle camera, changes in the driving environment can be perceived in real time, the impact of the external environment on driving can be accurately assessed, and rapid responses can be made.

[0021] The internal information collection module 13 is used to call the built-in sensor network to perform internal data collection and establish an internal information data set.

[0022] Specifically, the built-in sensor network refers to a series of sensors and devices pre-installed during driving training, which can continuously monitor the in-vehicle environment, the trainee's physiological state, and the dynamic behavior of the vehicle, usually including acceleration sensors, posture recognition sensors, facial recognition sensors, GPS sensors, temperature and humidity sensors, etc. Collect data from in-vehicle sensors and devices, and collect real-time data of trainees and vehicles by calling various sensors in the built-in sensor network, including the trainee's physiological state (such as facial expressions, eye movements, postures, etc.), the vehicle's motion state (such as speed, acceleration, steering angle, etc.), and in-vehicle environmental data (such as temperature, humidity, etc.). Internal data collection refers to obtaining various types of data about trainees and vehicles from the built-in sensor network.

[0023] The built-in sensor network includes at least facial perception sensors, posture recognition sensors, acceleration sensors, and positioning sensors. Among them, the facial recognition sensor is used to monitor the student's facial expressions, eye activities and pupil changes, and identify physiological states such as fatigue and distraction; the posture recognition sensor is used to determine the student's driving posture and judge whether there is an improper driving posture (such as leaning forward, leaning sideways, etc.); the acceleration sensor is used to monitor the vehicle's acceleration, deceleration and other dynamic states, and judge whether there are abnormal behaviors such as sudden braking and sudden acceleration; the GPS sensor is used to track the vehicle's position, speed and driving route, and analyze the student's specific route.

[0024] All collected information is integrated into a unified, structured internal information dataset, including information about trainees (such as facial expressions, postures), vehicles (such as movement status, speed), etc.

[0025] Through high-precision and high-stability internal data collection, the intelligent monitoring assistance system can more accurately evaluate the trainee's operating status and improve the accuracy of monitoring. Through internal information collection, various data of trainees and vehicles are obtained in real time to ensure timely understanding of the trainee's status and provide feedback, and accurately evaluate the trainee's driving behavior and safety.

[0026] The state extraction module 14 is used to extract the characteristics of the trainee state and the vehicle state from the internal information data set and establish an internal feature extraction result.

[0027] Specifically, the trainee status features are extracted from the internal information data set, that is, the physical and psychological status of the trainee during the training process, including the trainee's physiological data (such as facial expressions, fatigue level, mood, attention, etc.) and behavioral data (such as posture, movement coordination, driving habits, etc.). The facial expression data captured by the facial recognition sensor is used to extract information such as whether the trainee is tired and whether he is concentrating. The posture recognition sensor detects the trainee's driving posture, extracts what the trainee's driving posture is like, and whether there is an improper driving posture (such as leaning forward, leaning sideways, etc.). The facial perception sensor is used to track the trainee's eye movements, extract features such as the trainee's line of sight concentration and eye closure frequency, and judge whether the trainee is concentrating.

[0028] Extract vehicle status features from the internal information data set, that is, the dynamic information of the vehicle during driving, including the vehicle's motion state (such as acceleration, speed, steering angle, sudden braking, sudden acceleration, etc.) and driving route, etc. Use acceleration sensors and other vehicle dynamic sensors (such as gyroscopes, etc.) to extract information such as vehicle acceleration, steering angle, braking force, etc. Use GPS and positioning sensors to extract vehicle driving route features, including the student's driving path, speed changes in each section of the journey, steering changes, etc.

[0029] The feature information extracted from the student status and vehicle status is integrated to form a comprehensive feature extraction result, which includes the student's physiological and behavioral status and the vehicle's motion status information. The extracted feature data is preprocessed, including standardization, normalization, and removal of outliers to remove noise and abnormal values. The extracted features are analyzed to determine whether there are any abnormalities in the student's driving status. For example, the student's fatigue state (facial expression) is combined with incorrect driving posture (posture data), plus sudden braking and speeding behaviors (vehicle data). This information is integrated into a comprehensive feature extraction result, indicating that the student is at risk of fatigue driving, distraction, and improper driving behavior.

[0030] Through feature extraction, the trainee's physiological state (such as fatigue, distraction) and behavioral state (such as incorrect driving posture, sudden braking, speeding, etc.) are accurately assessed, and real-time analysis and evaluation are performed during the training process. If a trainee is found to have potential safety hazards (such as fatigue driving, dangerous driving behavior, etc.), a warning will be issued in time or the training plan will be adjusted to improve the trainee's driving safety.

[0031] The monitoring and early warning module 15 is used to call the initialized intelligent monitoring auxiliary system to perform joint abnormality authentication based on the internal feature extraction result and the external environment data set, and report a monitoring and early warning.

[0032] Specifically, the intelligent monitoring assistance system is a system that integrates multiple sensors and data analysis modules. It is designed to monitor the student's driving behavior and the environment in which he is located in real time, detect and evaluate potential driving anomalies, and provide real-time warnings or safety interventions. The initialized intelligent monitoring assistance system is called to perform joint anomaly authentication on the collected internal feature data and external environment data, and the student's driving status, vehicle status and external environment data are combined to determine whether there are abnormal conditions through comprehensive analysis. Joint anomaly authentication can more comprehensively and accurately identify risks and unsafe behaviors that may occur during driving. After joint authentication, the student's current driving status, vehicle motion status and external environmental conditions are evaluated to determine whether there are potential safety risks. If the joint authentication result shows that there is a risk, a monitoring warning is generated and a warning is sent to the student or coach. Once an abnormality is identified, the monitoring warning will be triggered in time and feedback will be given to the student and coach, and a warning message will be sent through the on-board audio equipment or on-board display. If the student fails to respond to the monitoring warning in time, further intervention will be carried out, such as system access to take over the vehicle, automatic deceleration or lane adjustment, etc., to ensure driving safety. Through joint anomaly authentication, the student's driving behavior, vehicle status and external environment are comprehensively considered to fully identify potential risks. This multi-level monitoring and authentication can more accurately determine whether there are safety hazards, thereby effectively improving driving safety. Joint abnormal authentication can more comprehensively and accurately identify abnormal situations by combining multi-dimensional data for analysis, thereby reducing the occurrence of false positives and missed positives.

[0033] Furthermore, the internal information collection module 13 in the trainee safety operation monitoring device for driving skill training is also used for:

[0034] Activate the facial perception sensor of the built-in sensor network, use the facial perception sensor to perceive the student's facial expression and eye activity, and establish a facial data set; activate the posture recognition sensor of the built-in sensor network, use the posture recognition sensor to perform driving posture recognition of the student, and establish a posture data set; integrate the facial data set and the posture data set into a student data set to construct an internal information data set.

[0035] Specifically, the built-in sensor network refers to the sensor system pre-installed in the car or driving training equipment, including facial perception sensors, posture recognition sensors, in-car temperature and humidity sensors, acceleration sensors, positioning sensors, etc., which can monitor the physical and psychological state of the trainee and the status of the vehicle in real time. Activate the facial perception sensors (such as infrared cameras, facial recognition cameras) in the built-in sensor network to collect the trainee's facial expressions and eye activity data, and monitor the trainee's eye movements (such as the focus position of the eyes), expression changes, and whether there are signs of fatigue in real time. By analyzing the trainee's facial expressions, eye movements, pupil changes, etc., the trainee's emotional state, fatigue level, or concentration level can be inferred. For example, the image recognition algorithm can be used to identify whether the trainee shows signs of drowsiness or fatigue (such as closing the eyes for a long time, blinking frequently, etc.). The student's facial image is captured by the camera, and the facial features are analyzed by the image processing algorithm. For example, yawning can be identified by detecting the degree and time of mouth opening; the blinking frequency can be calculated by detecting the opening and closing of the eyelids; signs of tension or distraction can be inferred by analyzing the direction of eye gaze, changes in pupil size, or the tension of facial muscles.

[0036] Activate the posture recognition sensor of the built-in sensor network to monitor the trainee's body posture and detect whether the trainee has an improper driving posture. Track the changes in the angle of the trainee's body to identify whether there is a bad posture such as leaning forward, leaning back, or leaning to the side. For example, the infrared sensor detects the trainee's body contour to determine whether he or she is sitting upright. The depth camera captures objects and people in three-dimensional space to accurately measure the distance between the trainee and the steering wheel and the angle of body inclination. The pressure sensor on the seat detects the trainee's body weight distribution to infer his or her sitting posture and body inclination. For example, if the trainee's body weight is mainly distributed on one side of the seat, it may indicate that his or her body is tilted.

[0037] The facial dataset and posture dataset are integrated into a comprehensive student dataset, which includes not only the student's facial status (such as fatigue, concentration, etc.), but also the student's body posture information (such as whether there is an improper driving posture). For example, if the student shows both a tired facial expression and a forward leaning posture, a comprehensive assessment is made that there are safety hazards and a warning is issued. The student's facial dataset, posture dataset, and data collected by other internal sensors (such as vehicle status, speed, braking force, etc.) are added to the internal information dataset to form a comprehensive profile of the student's behavior and physiological status. By combining facial expression and posture data, the student's status can be comprehensively assessed, and potential risk factors such as fatigue and distraction can be identified, thereby achieving more accurate safety monitoring.

[0038] Furthermore, the internal information collection module 13 in the trainee safety operation monitoring device for driving skill training is also used for:

[0039] Activate the acceleration sensor of the built-in sensor network, perceive the motion state of the vehicle based on the acceleration sensor, and establish a motion state data set; activate the positioning sensor of the built-in sensor network, perform driving route analysis, and establish a route data set; integrate the motion state data set and the route data set into a vehicle data set, and use the vehicle data set and the student data set to construct an internal information data set.

[0040] Specifically, the acceleration sensor of the built-in sensor network is activated to monitor the vehicle's motion state and measure the change in the vehicle's speed per unit time, that is, the acceleration, to evaluate the vehicle's motion state. Usually, the internal tiny mass block and spring are used to measure the acceleration change in three spatial directions. The acceleration sensor is composed of multiple accelerometers, which can sense the acceleration changes of the vehicle in different directions (such as front and back, left and right, up and down), and is used to monitor the dynamic behavior of the vehicle, such as rapid acceleration, sudden braking, and sharp turns. The vehicle's motion state data set is collected by the acceleration sensor, which contains the vehicle's dynamic behavior information such as acceleration, deceleration, and steering, and is used to evaluate the vehicle's motion state. The motion state data set can be used to determine whether the trainee is performing normal or abnormal driving operations, such as whether there is sudden braking or sudden acceleration.

[0041] Activate the positioning sensor (such as GPS) of the built-in sensor network to track the current location of the vehicle and perform driving route analysis. The positioning sensor records the vehicle's latitude and longitude, driving path, speed, driving direction and other data. The positioning sensor is usually combined with map data to provide accurate route navigation. Use the positioning sensor to perform driving route analysis and record the student's driving route, including the start and end locations, road section information, turn conditions, etc.

[0042] The motion state data set and route data set are integrated into the vehicle data set, and the vehicle data set is combined with the student data set to construct a complete internal information data set, which includes all relevant information such as the student's operating behavior, facial expressions, eye activities, driving posture, and the vehicle's motion state and driving route. By integrating the vehicle's motion state data and route data, we can fully understand the student's performance during driving and the vehicle's status, which helps to conduct more in-depth data analysis, such as analyzing the student's driving behavior on different road sections, the vehicle's dynamic response, and potential safety risks.

[0043] Further, as attached Figure 2 As shown, the monitoring and early warning module 15 in the trainee safety operation monitoring device for driving skill training is also used for:

[0044] The scene perception model within the intelligent monitoring assistance system is used to perceive and identify external environmental data and vehicle status characteristics, establish real-time danger scenarios, and identify danger levels; the real-time danger scenarios are used for processing and fitting, and a set of safe processing solutions and a set of dangerous processing solutions are established; the internal feature extraction results are used to perform trigger analysis on the safe processing solution set and the set of dangerous processing solutions, and joint abnormality authentication is performed based on the trigger analysis results and the danger level identification, and a monitoring warning is reported.

[0045] Specifically, through the built-in scene perception model of the intelligent monitoring assistance system, the external environment data is perceived and identified, and the roads, traffic signs, other vehicles, pedestrians and other objects in the current environment are identified and associated with the vehicle's motion state. At the same time, the current state of the vehicle is identified, including information such as vehicle movement and driving behavior, to evaluate whether the vehicle is driving normally or whether there is abnormal driving behavior. The scene perception model is a model for scene recognition and understanding based on external environmental data (such as images from on-board cameras, radar, lidar and other data). It perceives and classifies the external environment in real time, including identifying road signs, other vehicles, pedestrians, obstacles, traffic signals, etc., and evaluating potential dangerous scenes.

[0046] Based on the external environment data and vehicle status characteristics obtained through perception and recognition, real-time danger scenarios are established to identify scenarios that may pose a danger. Real-time danger scenarios are evaluated and different levels are assigned to different danger scenarios. Danger levels are usually classified according to factors such as the severity of the danger and the possible consequences, including high, medium, and low danger.

[0047] Processing and fitting of real-time dangerous scenarios, that is, generating safety processing solutions and emergency dangerous processing solutions for different safety scenarios. Processing and fitting means processing and matching different scenarios according to the characteristics of real-time dangerous scenarios to generate reasonable response solutions, not only considering the harmfulness of the scenarios, but also customizing personalized safety and dangerous processing solutions for each scenario based on factors such as the student's driving status and environmental conditions. For general driving scenarios, a series of preventive processing solutions are provided to help students optimize their driving behavior and avoid danger. For example, when it is detected that the student's vehicle has entered an area with an obstacle ahead, but the obstacle is far away and there is no direct danger, the safety processing solution set may suggest slowing down or adjusting the lane in advance. For sudden emergency dangerous situations, a series of emergency processing solutions are provided to help students respond in emergency situations and ensure safety. For example, if it is detected that the distance between the student's vehicle and the vehicle in front is too close and the other vehicle is braking suddenly, the emergency braking solution is activated to avoid danger.

[0048] According to the student status and vehicle status in the internal feature extraction results, trigger analysis is performed to determine whether specific safety or danger handling solutions need to be implemented. According to the student's driving behavior (such as emergency braking, over-acceleration) or changes in the external environment (such as the sudden appearance of obstacles), the corresponding safety or danger handling solutions are triggered. Combined with the trigger analysis results and the danger level identification, a joint analysis is performed, that is, the student's status (such as facial fatigue, abnormal posture), vehicle status (such as emergency braking, speeding, etc.) and external environment data (such as obstacles, traffic congestion, etc.) are jointly authenticated to determine whether there is an abnormal situation.

[0049] Joint anomaly authentication refers to the process of confirming anomalies by combining multiple data sources and analysis results. If an anomaly exists, an early warning is output according to the danger level. When an anomaly occurs, an early warning signal will be sent to the trainee through audio, screen or vibration to remind the trainee to take safety measures in time, such as slowing down, braking or adjusting the driving style. Through the comprehensive perception of the scene perception model and vehicle status, potential dangerous scenes can be identified in time, and early warnings can be issued to help trainees avoid accidents.

[0050] Furthermore, the monitoring and early warning module 15 in the trainee safety operation monitoring device for driving skill training is also used for:

[0051] The relative position between the real-time danger scene and the vehicle is obtained; a danger prompt is generated according to the relative position and the real-time danger scene, and the danger prompt is reported through voice and video display.

[0052] Specifically, a real-time danger scene is established through a scene perception model, and the relative position to the vehicle is determined through sensors (such as radar, laser radar, etc.). For example, the surrounding environment is scanned by laser radar to generate detailed three-dimensional point cloud data to help calculate the relative position to obstacles and other vehicles. The real-time danger scene is combined with the relative position of the vehicle to generate a danger prompt, that is, a warning message to remind the trainee to pay attention to the current potential danger. The prompt may be conveyed to the trainee through sound, image, vibration, etc., to help the trainee take appropriate driving actions in time (such as deceleration, avoidance, braking, etc.). A voice warning is generated through the on-board speaker to remind the trainee of the current danger. At the same time, on the on-board display screen, relevant danger information is displayed through a graphical interface, including displaying the icon of the vehicle in front and marking the relative distance between the trainee's vehicle and the distance change between the trainee's vehicle and the obstacle in front, providing a visual display of the danger area and identifying the emergency measures to be taken. Once a dangerous scene is detected and it is assessed that the relative position of the trainee's vehicle has a potential collision risk, the corresponding voice and video prompts are triggered, and the prompts will be adjusted according to the specific degree of danger. Through precise relative position data, accurate prompts are generated for each dangerous scenario to help students evaluate whether emergency response measures are needed, such as slowing down, braking or changing lanes. The combination of voice and video prompts can increase students' awareness of danger through multi-sensory reminders when students fail to perceive potential dangers in time.

[0053] Furthermore, the monitoring and early warning module 15 in the trainee safety operation monitoring device for driving skill training is also used for:

[0054] The response of the safety processing scheme is judged according to the route characteristics and vehicle status characteristics of the internal feature extraction results; if the response triggering the safety processing scheme is not detected within a preset time period, a vehicle-computer takeover prompt is generated; and the vehicle-computer takeover prompt is issued as a monitoring early warning alarm.

[0055] Specifically, the response of the safety handling plan is judged based on the route features and vehicle status features in the internal feature extraction results to determine whether it meets the preset safety standards and whether it is necessary to initiate the safety handling plan. Route features refer to various information related to the trainee's driving route, such as lane changes, road bends, the appearance of obstacles ahead, intersection conditions, etc., which can be used to determine whether the current driving route is safe and whether corresponding safety measures need to be taken (such as deceleration, lane changes, etc.). Vehicle status features refer to data related to the current state of the vehicle, such as the vehicle's speed, acceleration, braking status, steering angle, throttle pressure, etc., which reflect the dynamic changes of the vehicle and help determine whether there are potential driving risks.

[0056] A response time period is pre-set, that is, the trainee needs to respond within this time and make adjustments according to the safety handling plan. If the trainee is not detected to make a corresponding response (such as not slowing down, not braking, not changing lanes, etc.) within the preset time period, it is considered that the trainee has not responded and is ready to enter the emergency takeover state. The safety handling plan is a set of preventive measures or adjustment plans provided to the trainee by the system after evaluating the dangerous scene based on real-time data. It aims to help the trainee avoid potential dangers, such as ensuring driving safety by slowing down, changing lanes, braking, etc. Response discrimination refers to judging whether the current driving behavior meets the expected safety handling plan based on the set rules or algorithms. If the system detects that the trainee does not respond as expected (for example, does not slow down in time or does not change lanes), further emergency measures are initiated.

[0057] When it is detected that the trainee has not responded within the preset time and the situation has reached the safety crisis level (such as sudden braking in front or the distance of the obstacle is too close), the vehicle-computer takeover prompt is generated to warn the trainee that the on-board automatic driving system may need to take over the driving task. Once the vehicle-computer takeover prompt is generated, this information is issued as a monitoring warning and conveyed synchronously through multiple methods such as voice, video and vibration to ensure that the trainee can perceive and respond as soon as possible. At the same time, the on-board automatic driving system intervenes to perform emergency braking, steering and other operations to avoid accidents. The vehicle-computer takeover prompt usually includes voice warnings (such as please take braking measures immediately, the system will take over driving) and video prompts (such as displaying the current distance to dangerous objects, marking sudden braking or lane change instructions). By monitoring the trainee's driving behavior and vehicle status in real time, potential dangers can be identified in time, and appropriate safety handling solutions can be provided to the trainee to avoid accidents. When the trainee does not respond to the safety handling solution in time, the vehicle-computer takeover prompt and automatic takeover can effectively reduce the risks caused by the trainee's failure to respond and ensure driving safety.

[0058] Furthermore, the monitoring and early warning module 15 in the trainee safety operation monitoring device for driving skill training is also used for:

[0059] The student characteristics of the internal feature extraction result are obtained, and the student status abnormality is identified according to the student characteristics; an auxiliary abnormality authentication is generated according to the student status abnormality identification result; after compensating the joint abnormality authentication according to the auxiliary abnormality authentication, an updated monitoring warning is reported.

[0060] Specifically, the student characteristics of the internal feature extraction results are obtained, and the student status is monitored to determine whether there are abnormal conditions, including fatigue driving, distracted driving, slow reaction, etc., which may affect the student's driving decision-making, thereby increasing the risk of traffic accidents. For example, by analyzing the student's facial expressions and eye movement data, it is possible to identify whether the student has signs of fatigue driving; by analyzing whether the student frequently turns his head, uses a mobile phone, etc., it is possible to identify whether the student is distracted.

[0061] Based on the abnormal identification results of the trainee's state, auxiliary abnormal authentication information is generated, that is, additional authentication generated based on the abnormal identification results of the trainee's state, which is used to supplement the joint abnormal authentication. For example, if the trainee is identified as being in a fatigued state, the auxiliary abnormal authentication may include the fatigue level and recommended countermeasures. The auxiliary abnormal authentication information is combined with the existing joint abnormal authentication results, and the joint abnormal authentication is compensated by the auxiliary abnormal authentication, that is, the joint abnormal authentication is adjusted and corrected to ensure that the abnormal authentication results are more accurate. Even when the vehicle state is relatively stable and the road conditions are good, the trainee's fatigue state is still analyzed, and it is considered that the trainee's abnormality is more serious at this time. Combined with the abnormalities in vehicle speed and posture, an emergency warning is initiated.

[0062] According to the supplemented joint abnormal authentication results, an updated monitoring warning is generated, and a warning is issued in time through voice prompts or video prompts, and measures such as automatic deceleration may be taken. By comprehensively analyzing the multi-dimensional data of the trainees (face, eye movements, posture, etc.), it is accurately judged whether the trainees have abnormal driving conditions such as fatigue and distraction, reducing false alarms and missed alarms, and issuing warnings in time or taking automatic intervention measures when the trainees are in abnormal conditions, helping trainees adjust their driving behavior and avoid accidents caused by fatigue driving, distracted driving, etc.

[0063] Furthermore, the trainee safety operation monitoring device for driving skill training also includes a feedback module, which is used to:

[0064] Data mapping is performed on the internal information data set and the external environment data set, and an operation abnormality report is generated, and the operation abnormality report is fed back to the trainee coach.

[0065] Specifically, the feedback module is a key part of data used to collect, record and feedback driving behavior, vehicle status, environmental information, etc., to ensure that the trainee's safe operation monitoring device can provide the instructor with real-time or regular feedback on the trainee's driving performance, especially warnings and reports on abnormal operations. The internal information data set and the external environment data set are mapped and associated through data such as timestamps or location coordinates to ensure that each data point corresponds to a specific driving scenario. The purpose is to compare the trainee's operating behavior with changes in the external environment to identify potential operational anomalies. Based on the data mapping, the trainee's driving behavior is analyzed and possible abnormal operations are identified.

[0066] Generate an abnormal operation report, detailing the time, location, student's operation, vehicle status, and possible environmental factors of the abnormal event. The generated abnormal operation report is sent to the student's coach via email, SMS or other communication methods. The coach can evaluate the student's driving behavior based on the report content and provide targeted guidance and training to help the student enhance driving skills and safety awareness. Through real-time monitoring and data feedback of each student's driving behavior, the coach can provide students with more targeted teaching feedback, helping students quickly improve irregular driving behavior through specific feedback, and ensuring that the student's driving behavior meets safety standards.

[0067] In summary, the student safety operation monitoring device for driving skill training provided by this application has the following technical effects:

[0068] The initialization module is used to obtain the trainee's training data, and the system attention initialization of the intelligent monitoring assistance system is performed based on the training data. The training data includes the trainee's historical training data, current training scene data, and current training subject data; the external information acquisition module is used to call the external vehicle-mounted camera to perform real-time environmental data acquisition and establish an external environment data set; the internal information acquisition module is used to call the built-in sensor network to perform internal data acquisition and establish an internal information data set; the state extraction module is used to extract the characteristics of the trainee's state and vehicle state from the internal information data set and establish an internal feature extraction result; the monitoring and early warning module is used to call the initialized intelligent monitoring assistance system to perform joint abnormal authentication based on the internal feature extraction result and the external environment data set, and report a monitoring and early warning. In other words, by integrating the trainee's historical training data, current training scene data, and current training subject data, collecting real-time environment and internal data, extracting the characteristics of the trainee and vehicle status, and combining the joint abnormal authentication to perform proofreading and verification in multiple dimensions, the accuracy of safe operation monitoring is improved, ensuring the trainee's safe operation, thereby improving the safety of road traffic.

[0069] The above description of the disclosed embodiments enables those skilled in the art to implement or use the present application. Various modifications to these embodiments will be apparent to those skilled in the art, and the general principles defined herein may be implemented in other embodiments without departing from the spirit or scope of the present application. Therefore, the present application will not be limited to the embodiments shown herein, but will conform to the widest scope consistent with the principles and novel features disclosed herein.

[0070] Obviously, those skilled in the art can make various changes and modifications to the present application without departing from the spirit and scope of the present application. Thus, if these modifications and variations of the present application belong to the scope of the present application and its equivalent technology, the present application is also intended to include these modifications and variations.

Claims

1. A student safety operation monitoring device for driving skills training, characterized in that: The student safety operation monitoring device for driving skill training comprises: An initialization module is used to obtain the trainee's training data and perform system attention initialization of the intelligent monitoring assistance system based on the training data, wherein the training data includes the trainee's historical training data, current training scenario data, and current training subject data; The external information collection module is used to call the external vehicle-mounted camera to perform real-time environmental data collection and establish an external environment data set; An internal information collection module is used to call the built-in sensor network to perform internal data collection and establish an internal information data set; A state extraction module, used to extract features of the trainee state and the vehicle state from the internal information data set, and establish an internal feature extraction result; The monitoring and early warning module is used to call the initialized intelligent monitoring auxiliary system to perform joint abnormality authentication based on the internal feature extraction result and the external environment data set, and report a monitoring and early warning.

2. The student safety operation monitoring device for driving skill training according to claim 1, characterized in that: The internal information acquisition module is used for: activating the facial perception sensor of the built-in sensor network, using the facial perception sensor to perceive the facial expression and eye movement of the trainee, and establishing a facial data set; activating a posture recognition sensor of the built-in sensor network, using the posture recognition sensor to perform driving posture recognition of the trainee, and establishing a posture data set; The facial dataset and the posture dataset are integrated into a student dataset to construct an internal information dataset.

3. The student safety operation monitoring device for driving skill training according to claim 2, characterized in that: The internal information acquisition module is also used for: activating an acceleration sensor of the built-in sensor network, sensing the motion state of the vehicle based on the acceleration sensor, and establishing a motion state data set; activating a positioning sensor of the built-in sensor network, performing driving route analysis, and establishing a route data set; The motion state data set and the route data set are integrated into a vehicle data set, and the vehicle data set and the student data set are used to construct an internal information data set.

4. The student safety operation monitoring device for driving skill training according to claim 1, characterized in that: The monitoring and early warning module is used for: Through the scene perception model in the intelligent monitoring assistance system, the external environment data and vehicle status characteristics are perceived and identified, real-time danger scenes are established, and danger levels are marked; Using the real-time dangerous scenario to perform processing fitting, establishing a safe processing solution set and a dangerous processing solution set; The internal feature extraction results are subjected to trigger analysis of a safety processing solution set and a danger processing solution set, and joint abnormality authentication is performed based on the trigger analysis results and the danger level identification, and a monitoring warning is reported.

5. The student safety operation monitoring device for driving skill training according to claim 4, characterized in that: The monitoring and early warning module is also used for: Obtaining the relative position of the real-time dangerous scene and the vehicle; A danger prompt is generated according to the relative position and the real-time danger scene, and the danger prompt is reported through voice and video display.

6. The student safety operation monitoring device for driving skill training according to claim 4, characterized in that: The monitoring and early warning module is also used for: Perform response judgment of the safety processing solution according to the route characteristics and vehicle status characteristics of the internal feature extraction result; If no response is detected to trigger the safety processing solution within the preset time period, a vehicle computer takeover prompt is generated; The vehicle computer takeover prompt is issued as a monitoring early warning alarm.

7. The student safety operation monitoring device for driving skill training according to claim 4, characterized in that: The monitoring and early warning module is also used for: Acquire the student characteristics of the internal feature extraction result, and identify the abnormal status of the student according to the student characteristics; Generate auxiliary abnormality certification based on the abnormal identification results of the student status; After compensating the combined abnormal authentication according to the auxiliary abnormal authentication, an updated monitoring warning is reported.

8. The student safety operation monitoring device for driving skill training according to claim 1, characterized in that: The student safety operation monitoring device for driving skill training also includes: The feedback module is used to perform data mapping and recording on the internal information data set and the external environment data set, and to generate an operation abnormality report, and to feed back the operation abnormality report to the trainee coach.

Images