A software for judging driving skills of trainees by a camera recognition system

Abstract

The present application relates to the technical field of judging the driving skills of trainees, and provides a software for judging the driving skills of trainees through a camera recognition system, which comprises a hardware support module and a software function module; the hardware support module comprises a multi-view image acquisition unit and an auxiliary sensing unit; the software function module comprises a feature recognition unit, a data processing unit, a calibration training unit, a VR simulation unit, a communication interaction unit and a report generation unit; the multi-view image acquisition unit adopts a 4-way machine vision module and is respectively installed at the center of the front bumper, the center of the rear bumper and below the left and right rearview mirrors of the vehicle. Through a deflection angle correction formula, the distance calculation error is reduced from >5 cm in the prior art to ≤2 cm, and the key and subtle operation of accurately identifying the edge line distance deviation of 2 cm can be accurately recognized. After 12 or more demonstration data fitting, the deviation recognition sensitivity is improved compared with single demonstration benchmark, and the problem of misjudgment caused by standard confusion is effectively avoided.

Description

Technical Field

[0001] This invention relates to the field of software technology for judging driving skills, specifically to software that uses a camera recognition system to judge driving skills. Background Technology

[0002] In the current driving test training field, camera-based software for judging students' driving skills has become a core tool to assist manual teaching. It is mainly used in the second subject's field training and the third subject's basic road training. Its core logic is to use the vehicle-mounted camera to collect images of the vehicle's surrounding environment, and combine simple feature recognition and data comparison to judge the student's mastery of skills such as controlling the distance to the edge line, the standardization of driving trajectory, and the timing of operation. It aims to alleviate the contradiction of "few instructors and a large number of students" in traditional teaching and promote the standardization of the training process.

[0003] However, existing technologies have multiple shortcomings that make it difficult to meet the needs of precise training: image acquisition and feature recognition capabilities are insufficient, most software uses single-view cameras, which do not fully cover the sides and front edges of the vehicle, and lack an active adjustment mechanism. When the vehicle is bumpy or there are stains on the edges, the edge recognition miss rate is high; at the same time, the distance calculation does not take into account the vehicle's skew angle correction, and only obtains the result through simple pixel conversion. The actual error often exceeds 5cm, and it cannot distinguish subtle operational problems that affect the test results, such as "edge distance deviation of 2cm". Summary of the Invention

[0004] To address the shortcomings of existing technologies, this invention provides software that uses a camera recognition system to assess students' driving skills. This solves the problems at the student level, where untimely feedback and inaccurate assessments lead to repeated reinforcement of incorrect operations, resulting in excessively long average training times for the second subject. At the instructor level, the software cannot provide quantitative data on students' weaknesses, requiring repeated review of trajectory reports to pinpoint problems, thus hindering the improvement of teaching efficiency.

[0005] To achieve the above objectives, the present invention provides the following technical solution: a software for judging a student's driving skills through a camera recognition system, comprising a hardware support module and a software function module: the hardware support module includes a multi-view image acquisition unit and an auxiliary sensing unit, and the software function module includes a feature recognition unit, a data processing unit, a calibration and training unit, a VR simulation unit, a communication interaction unit, and a report generation unit; The multi-view image acquisition unit uses a 4-channel machine vision module, which is installed in the center of the front bumper, the center of the rear bumper, and below the left and right rearview mirrors, respectively, and works with a two-degree-of-freedom servo gimbal to achieve multi-angle image capture. The auxiliary sensing unit includes an ultrasonic sensor with a range of 0.3~2.5m and an accuracy of ±0.2cm, and an tilt sensor with a range of -180°~180° and an accuracy of ±0.05°. The servo gimbal of the multi-view image acquisition unit is equipped with an anti-shake sub-unit. When the auxiliary sensing unit detects an instantaneous change in the vehicle's yaw angle > 3° / s, the anti-shake sub-unit adjusts the servo speed at a frequency of 15Hz using a PID algorithm to ensure that the edge line is always in the center area of ​​the image.

[0006] Preferably, the feature recognition unit is configured with a missed detection compensation subunit. When the edge line feature is not recognized in three consecutive frames of images, the missed detection compensation subunit controls the servo gimbal to perform a 90° up-and-down sweep at a speed of 10° / s until the edge line is recaptured. During the sweeping process, historical edge line data is called for temporary fitting to avoid interruption of the judgment.

[0007] Preferably, in the correction formula of the data processing unit, the distance between the front and rear wheels of the vehicle is... It supports user-defined input, and the system automatically updates the distance calculation parameters after input, with an adaptation response time of ≤1s.

[0008] Preferably, the standard database configuration of the calibration training unit includes a dynamically updated sub-unit. When the color of the training field sidelines changes or key points are adjusted, the dynamically updated sub-unit allows the coach to upload three new demonstration data points via the coach client, automatically fitting and updating the data. The update process does not require disassembling the hardware and takes ≤5 minutes.

[0009] Preferably, the VR simulation unit is equipped with a scene parameter adjustment subunit, which allows users to adjust the light intensity, rain and snow density, and road surface friction coefficient of the virtual scene, and can simulate more than 20 subdivided working conditions to meet the needs of driving test scenarios in different regions.

[0010] Preferably, the student client of the communication interaction unit is configured with a real-time reminder subunit, which alerts the student when their operation deviation exceeds [a certain threshold]. When the deviation is ±3cm or the turning delay is >0.5s, the real-time reminder subunit will provide a synchronous reminder via voice and pop-up window, with a reminder response time ≤0.3s.

[0011] Preferably, the coach client of the communication interaction unit is configured with a weakness analysis subunit, which can count the deviation data of the trainee in the last 10 training sessions, identify the operation item with the highest deviation rate, and generate a guidance list containing improvement plans. The list can be exported as a PDF.

[0012] Preferably, the report generation unit is configured with a historical comparison subunit, which can store nearly 60 training reports from trainees, supports filtering by "training date" and "project type", and displays the trend of deviation rate changes through a line graph to help trainees intuitively view their skill improvement trajectory.

[0013] Preferably, the multi-view image acquisition unit is also configured with an image stitching subunit, which stitches together a 360° panoramic image based on the optimized intrinsic and extrinsic parameters of the camera, with a distortion rate of ≤1% after stitching, allowing trainees to view the panoramic environment around the vehicle in real time.

[0014] The working steps of a software that uses a camera recognition system to determine a learner's driving skills are as follows: Step 1: The VR simulation unit generates various virtual working conditions such as sunny days, rainy or snowy days, and nighttime through mathematical modeling and 3D reconstruction, and outputs virtual images to the image acquisition unit; Step 2: The feature recognition unit extracts the training field edge lines, key location points, and vehicle contours by combining color thresholding with contour recognition algorithms; Step 3: The data processing unit first uses formulas... Calculate the initial distance, and then combine it with the vehicle swerve angle. By correction formula To obtain the precise distance, among which The distance between the front and rear wheels of the vehicle is set to a value of 2.5~3.2m, and the student's driving trajectory data is generated simultaneously. Step 4: The calibration training unit collects data through n≥12 demonstration operations by the coach to obtain the standard distance range of each key location point. And compare it with student data in real time; Step 5: The communication interaction unit enables data interaction with the student client and the coach client via Bluetooth 5.2 or WiFi 6; Step Six: The report generation unit outputs the deviation rate. Training reports outlining weaknesses and suggestions for improvement.

[0015] This invention provides software that uses a camera recognition system to assess a learner's driving skills. It offers the following advantages: 1. This invention reduces the distance calculation error from >5cm to ≤2cm in the prior art through the skew angle correction formula. It can accurately identify critical and subtle operations such as edge distance deviation of 2cm. The standard database has been fitted with more than 12 demonstration data, and the deviation identification sensitivity is improved compared with a single demonstration benchmark, effectively avoiding the problem of misjudgment caused by standard confusion.

[0016] 2. The VR simulation unit of this invention can cover more than 20 subdivided working conditions. Compared with the traditional calibration method that relies on real working conditions, the scene coverage is improved and the calibration efficiency is improved. There is no need to build a special simulated rain and snow weather site, which reduces the related operating costs of driving schools by 45%. At the same time, it is compatible with the custom parameter function of multiple vehicle models.

[0017] 3. The real-time reminder sub-unit of this invention has a response time of ≤0.3s, which can correct student operation errors instantly, avoid error solidification, shorten the average training time of students in Subject 2, and improve training efficiency; the weak point analysis sub-unit of the coach client can automatically locate student problems, and increase the number of students that the coach can effectively manage in a single session. Attached Figure Description

[0018] Figure 1 This is a system diagram of the present invention. Detailed Implementation

[0019] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0020] Example: Please see the appendix Figure 1 This invention provides software for judging a student's driving skills through a camera recognition system, including a hardware support module and a software function module: the hardware support module includes a multi-view image acquisition unit and an auxiliary sensing unit, and the software function module includes a feature recognition unit, a data processing unit, a calibration training unit, a VR simulation unit, a communication interaction unit, and a report generation unit; The multi-view image acquisition unit uses a 4-channel machine vision module, which is installed in the center of the front bumper, the center of the rear bumper, and below the left and right rearview mirrors, respectively, and works with a two-degree-of-freedom servo gimbal to achieve multi-angle image capture. The auxiliary sensing unit includes an ultrasonic sensor with a range of 0.3~2.5m and an accuracy of ±0.2cm, and an tilt sensor with a range of -180°~180° and an accuracy of ±0.05°. The working steps of the software functional module are as follows: Step 1: The VR simulation unit generates various virtual working conditions such as sunny days, rainy / snowy days, and nighttime through mathematical modeling and 3D reconstruction, and outputs virtual images to the image acquisition unit. For example, the reversing into a parking space scenario in driving test 2 (parking space length 5.25m, parking space width 2.3m, edge line width 15cm) is configured with detailed parameters: sunny scene light intensity 8000 lux, road surface friction coefficient 0.8; rainy / snowy scene visibility 300m, rain / snow density 12 drops / m²·s, light intensity 1500 lux; nighttime scene light intensity 100 lux, equipped with simulated street light spot (diameter 1.5m, light intensity 300 lux). The virtual image output frame rate is synchronized with the machine vision module acquisition frame rate (30fps) to avoid feature misalignment caused by timing deviation. Step 2: The feature recognition unit extracts the training field edges, key position points, and vehicle contours using a combination of color thresholding and contour recognition algorithms. Specifically, it calls the OpenCV image processing library, first filtering edges based on the HSV color space (yellow edges: H25~35, S120~200, V100~220; white edges: H0~10, S0~40, V200~255), then extracting continuous contours of the edges using the Canny edge detection algorithm (low threshold 50, high threshold 150), and simultaneously identifying key position points (marked with black 10cm×10cm blocks, HSV thresholds H0~10, S0~20, V0~50, such as corners for reversing into a parking space, and parking lines for parallel parking). For the vehicle contours, the gray-level difference method (gray-level difference between the vehicle body and the background ≥80) is used to locate the coordinates of the front and rear vertices. The feature extraction time for each frame is ≤30ms, meeting the real-time judgment requirements. Step 3: The data processing unit first uses formulas... Calculate the initial distance, and then combine it with the vehicle swerve angle. By correction formula To obtain the precise distance, among which The distance between the front and rear wheels of the vehicle is set to a value of 2.5~3.2m, and the driving trajectory data of the trainee is generated simultaneously, using a compact sedan as an example. Taking a 2.6m reverse parking maneuver as an example, the ultrasonic sensor measures d=1.1m, and the tilt sensor measures... =28°, calculate the initial distance If the vehicle deflects at an angle Substituting into the correction formula, we get This refers to the precise distance between the vehicle's rear wheels and the edge line; the driving trajectory data records the vehicle coordinates (based on a coordinate system established in a virtual scene: the X-axis is along the edge line, and the Y-axis is perpendicular to the edge line) at a sampling frequency of 10Hz, with a trajectory point spacing of ≤0.3m, which can capture subtle operations such as "steering delay of 0.5s"; Step 4: The calibration training unit collects data through n≥12 demonstration operations by the coach to obtain the standard distance range of each key location point. The data is compared in real time with the trainees' data. Specifically, before the trainees' training, the instructor completes 15 standardized demonstrations. The system uses MATLAB's least squares method to fit the 15 sets of demonstration data (each set containing 8 key parameters such as the distance between the car's front and the parking space corner, and the distance between the car's rear and the edge line) to obtain... Examples include "the front of the car is 32±2cm from the left corner of the parking space" and "the rear of the car is 28±2cm from the right edge of the parking space." During training, the unit compares the trainees' performance in real time. and A deviation ≤ 2cm is marked as “normal”, a deviation ≤ 3cm is marked as “minor deviation”, and a deviation > 3cm is marked as “serious deviation”. Step 5: The communication interaction unit communicates with student and coach clients via Bluetooth 5.2 or WiFi 6. Bluetooth 5.2 has a transmission rate of 2Mbps and a communication latency of ≤0.3s, supporting one-to-one connections (student clients); WiFi 6 has a transmission rate of 1.2Gbps and a communication latency of ≤0.1s, supporting multiple device access (≤8 coach clients online simultaneously). Data transmission uses AES-128 encryption to ensure that training data is not leaked. Step Six: The report generation unit outputs the deviation rate. Training reports should include weaknesses and suggestions for improvement. For example, after a student completes 10 reversing maneuvers into a parking space, the report should record "2 serious deviations (rear of the car exceeding the edge line) and 5 minor deviations (front of the car too close to the corner of the parking space)" and calculate the deviation rate. (The denominator is "number of training sessions × number of key parameters per session"), and the weak point is identified as "the rear of the car is delayed by 0.4~0.6 seconds due to steering wheel turning". The improvement suggestion is "when the lower edge of the left rearview mirror is aligned with the parking line, immediately turn the steering wheel fully to the right, and control the steering wheel turning time within 0.3 seconds". The report can be exported as a PDF. Specifically, the workflow of the software's functional modules is deeply adapted to actual driving test training scenarios. For example, for the parallel parking exercise in Subject 2, the VR simulation unit can be additionally configured with a "damaged edge line" scenario (simulating wear and tear on the edge lines of a real training field, with a damaged section length of 50cm). The feature recognition unit ensures continuous recognition through a contour completion algorithm (fitting the damaged section based on the slope of adjacent edge lines). The data processing unit automatically adapts and corrects the formula parameters for SUV models, eliminating the need for manual adjustments. The calibration training unit supports instructors updating the software according to regional test differences. To meet localized needs.

[0021] The multi-view image acquisition unit's servo gimbal is equipped with an anti-shake subunit. When the auxiliary sensing unit detects an instantaneous change in vehicle yaw angle >3° / s, the anti-shake subunit adjusts the servo speed at a frequency of 15Hz using a PID algorithm to ensure the edge line remains in the center of the image. The feature recognition unit is equipped with a missed detection compensation subunit. When the edge line feature is not recognized in three consecutive frames, the missed detection compensation subunit controls the servo gimbal to perform a 90° up-and-down sweep at a speed of 10° / s until the edge line is recaptured. During the sweep, historical edge line data is used for temporary fitting to avoid interruption of the judgment. In the correction formula of the data processing unit, the distance between the front and rear wheels of the vehicle is considered. Supports user-defined input; the system automatically updates distance calculation parameters after input, with a response time of ≤1s. The standard database configuration for calibrating training units dynamically updates sub-units. When the color of the training field's sidelines changes or key points are adjusted, the dynamic sub-unit update supports coaches uploading three new demonstration data points via the coach client, automatically fitting and updating the data. The update process requires no hardware disassembly and takes ≤5 minutes. The VR simulation unit is equipped with a scene parameter adjustment subunit, allowing users to adjust the virtual scene's light intensity, rain and snow density, and road surface friction coefficient. It can simulate more than 20 subdivided working conditions to meet the needs of driving test scenarios in different regions. The student client of the communication interaction unit is equipped with a real-time reminder subunit. When the student's operation deviation exceeds... When the deviation is ±3cm or the steering delay is >0.5s, the real-time reminder subunit provides synchronous reminders via voice and pop-up window, with a reminder response time ≤0.3s. The coach client of the communication interaction unit is configured with a weakness analysis subunit, which can count the deviation data of the trainee in the last 10 training sessions, identify the operation item with the highest deviation rate, and generate a guidance list with improvement plans. The list can be exported as PDF format. The report generation unit is configured with a historical comparison subunit, which can store the trainee's last 60 training reports, supports filtering by "training date" and "project type", and displays the trend of deviation rate changes through line graphs to help trainees intuitively view the trajectory of skill improvement. The multi-view image acquisition unit is also configured with an image stitching subunit, which stitches together a 360° surround view image based on optimized camera intrinsic and extrinsic parameters, with a distortion rate of ≤1% after stitching, allowing trainees to view the panoramic environment around the vehicle in real time. Specifically, the actual application details of each functional subunit are as follows: The anti-shake subunit reduces the servo motor speed from 5° / s to 2° / s using a PID algorithm when the vehicle passes a 5cm high speed bump on the training track, ensuring that the edge line deviates from the image center by ≤3 pixels; the missed detection compensation subunit controls the servo motor gimbal to sweep 90° when there is 10cm of dirt on the edge line, while simultaneously using edge line data from the previous 5 frames to fit a temporary edge line, avoiding a judgment interruption exceeding 100ms; when the training track edge line changes from yellow to white, the instructor uploads 3 parallel parking demonstration data sets, and the subunit automatically updates the data to fit the white edge line. The VR scene parameter adjustment subunit supports setting "snowy weather" for driving schools in the north and "plum rain weather" for driving schools in the south; the real-time reminder subunit issues an 85dB voice prompt within 0.3 seconds when the student's wheel deviation from the edge line is 4cm during a hill start, "The wheel is too close to the edge line, adjust the direction slightly to the right by 10°", and simultaneously displays the deviation value in a pop-up window; the instructor client weakness analysis subunit statistically analyzes the student's data from the last 10 hill starts, identifies "edge line deviation rate of 60%", and generates a guidance list "When fixing the position on the hill, align the lower edge of the right rearview mirror with the edge line, and control the clutch lifting speed at 0.5cm / s", and the list supports adding instructor annotations and exporting to PDF; the historical comparison subunit stores 60 training reports for students, and students can filter "reverse parking in the last month" data to view the trend of the deviation rate decreasing from 35% to 10% through a line graph; the image stitching subunit stitches 360° surround view images based on optimized camera parameters, and students can switch to "rear view" to confirm the distance to the parking space corner.

[0022] Although embodiments of the invention have been shown and described, it will be understood by those skilled in the art that various changes, modifications, substitutions and alterations can be made to these embodiments without departing from the principles and spirit of the invention, the scope of which is defined by the appended claims and their equivalents.

Claims

1. Software for judging a student's driving skills using a camera recognition system, characterized in that, It includes a hardware support module and a software function module: the hardware support module includes a multi-view image acquisition unit and an auxiliary sensing unit, and the software function module includes a feature recognition unit, a data processing unit, a calibration and training unit, a VR simulation unit, a communication interaction unit, and a report generation unit; The multi-view image acquisition unit uses a 4-channel machine vision module, which is installed in the center of the front bumper, the center of the rear bumper, and below the left and right rearview mirrors, respectively, and works with a two-degree-of-freedom servo gimbal to achieve multi-angle image capture. The auxiliary sensing unit includes an ultrasonic sensor with a range of 0.3~2.5m and an accuracy of ±0.2cm, and an tilt sensor with a range of -180°~180° and an accuracy of ±0.05°. The servo gimbal of the multi-view image acquisition unit is equipped with an anti-shake sub-unit. When the auxiliary sensing unit detects an instantaneous change in the vehicle's yaw angle > 3° / s, the anti-shake sub-unit adjusts the servo speed at a frequency of 15Hz using a PID algorithm to ensure that the edge line is always in the center area of ​​the image.

2. The software for judging a student's driving skills using a camera recognition system according to claim 1, characterized in that, The feature recognition unit is equipped with a missed detection compensation subunit. When the edge line feature is not recognized for three consecutive frames of images, the missed detection compensation subunit controls the servo gimbal to perform a 90° up-and-down sweep at a speed of 10° / s until the edge line is recaptured. During the sweeping process, historical edge line data is called up for temporary fitting to avoid interruption of the judgment.

3. The software for judging a student's driving skills using a camera recognition system according to claim 1, characterized in that, In the correction formula of the data processing unit, the distance between the front and rear wheels of the vehicle... It supports user-defined input, and the system automatically updates the distance calculation parameters after input, with an adaptation response time of ≤1s.

4. The software for judging a student's driving skills using a camera recognition system according to claim 1, characterized in that, The standard database configuration of the calibration training unit is dynamically updated. When the color of the training field sidelines changes or key points are adjusted, the dynamically updated subunit allows the coach to upload three new demonstration data sets via the coach client, and automatically fits and updates the data. The update process does not require disassembling the hardware and takes ≤5 minutes.

5. The software for judging a student's driving skills using a camera recognition system according to claim 1, characterized in that, The VR simulation unit is equipped with a scene parameter adjustment subunit, which allows users to adjust the light intensity, rain and snow density, and road friction coefficient of the virtual scene. It can simulate more than 20 subdivided working conditions to meet the needs of driving test scenarios in different regions.

6. The software for judging a student's driving skills using a camera recognition system according to claim 1, characterized in that, The student client of the communication interaction unit is configured with a real-time reminder subunit. When the student's operation deviation exceeds... When the deviation is ±3cm or the turning delay is >0.5s, the real-time reminder subunit will provide a synchronous reminder via voice and pop-up window, with a reminder response time ≤0.3s.

7. The software for judging a student's driving skills using a camera recognition system according to claim 1, characterized in that, The coach client of the communication interaction unit is configured with a weakness analysis subunit, which can count the deviation data of the trainee in the last 10 training sessions, identify the operation item with the highest deviation rate, and generate a guidance list with improvement plans. The list can be exported as a PDF.

8. The software for judging a student's driving skills using a camera recognition system according to claim 1, characterized in that, The report generation unit is equipped with a historical comparison subunit, which can store nearly 60 training reports from trainees. It supports filtering by "training date" and "project type" and displays the trend of deviation rate changes through a line graph, helping trainees to intuitively view their skill improvement trajectory.

9. The software for judging a student's driving skills using a camera recognition system according to claim 1, characterized in that, The multi-view image acquisition unit is also equipped with an image stitching subunit, which stitches together 360° panoramic images based on optimized camera intrinsic and extrinsic parameters. The image distortion rate after stitching is ≤1%, allowing trainees to view the panoramic environment around the vehicle in real time.

10. A set of operating steps for software that uses a camera recognition system to determine a student's driving skills, comprising using the software described in any one of claims 1-9, characterized in that... The working steps of the software functional module are as follows: Step 1: The VR simulation unit generates various virtual working conditions such as sunny days, rainy or snowy days, and nighttime through mathematical modeling and 3D reconstruction, and outputs virtual images to the image acquisition unit; Step 2: The feature recognition unit extracts the training field edge lines, key location points, and vehicle contours by combining color thresholding with contour recognition algorithms; Step 3: The data processing unit first uses formulas... Calculate the initial distance, and then combine it with the vehicle swerve angle. By correction formula To obtain the precise distance, among which The distance between the front and rear wheels of the vehicle is set to a value of 2.5~3.2m, and the student's driving trajectory data is generated simultaneously. Step 4: The calibration training unit collects data through n≥12 demonstration operations by the coach to obtain the standard distance range of each key location point. And compare it with student data in real time; Step 5: The communication interaction unit enables data interaction with the student client and the coach client via Bluetooth 5.2 or WiFi 6; Step Six: The report generation unit outputs the deviation rate. Training reports outlining weaknesses and suggestions for improvement.

Images