A subject three vehicle test examiner behavior intelligent supervision method and system

Abstract

The application discloses a subject three vehicle test examiner behavior intelligent supervision method and system, and belongs to the technical field of driving test supervision. The method realizes intelligent supervision on the whole process of the vehicle test examiner through the steps of vehicle test examiner filing, random vehicle allocation before the test, identity verification, multi-modal behavior monitoring during the test, real-time early warning and post-test traceability analysis. The system contains seven modules of filing management, random vehicle allocation, identity verification, multi-modal data acquisition, behavior analysis, early warning disposal and data analysis, can accurately identify 28 specific irregular behaviors under 17 test items such as simulated light, starting and parking on the side, reduces data redundancy through slicing storage, and constructs a vehicle test examiner behavior portrait through multi-dimensional data analysis. The application solves the problems of hidden behavior, difficult evidence collection and large police consumption in the traditional supervision mode, effectively curbs the phenomena of assisting cheating and malicious interference in the test, and improves the public credibility of the driving test.

Description

Technical Field

[0001] This invention belongs to the field of driver's license examination supervision technology, and specifically relates to a method and system for intelligent supervision of the behavior of examiners accompanying the examiners in the third subject. Background Technology

[0002] Currently, the driving test for motor vehicles generally uses public roads as the test route. In order to ensure the safety of candidates and public transportation, each test must be accompanied by an examiner (also known as a safety officer) throughout the entire test.

[0003] While this mechanism reduces security risks during the examination process from a physical perspective, the closed environment of the examination vehicle, which only contains the examinee and the accompanying examiner, means that the examiner's behavior can directly interfere with the fairness of the examination, a problem that has long existed.

[0004] Cases reported by traffic management departments across the country in recent years show frequent instances of examiners assisting in cheating and maliciously interfering with test results. Some examiners use concealed hand gestures to instruct candidates to adjust the vehicle's position during the parallel parking maneuver, tap on the window to prompt candidates during lane changes, and even directly apply the secondary brake while the candidate is driving normally, causing them to fail the test. Such behavior not only undermines the core principles of fairness, openness, and impartiality in examinations but also triggers numerous complaints and disputes from candidates, seriously damaging the credibility of traffic management departments.

[0005] Further analysis of the pain points of the existing regulatory model reveals three core issues: First, the closed testing environment creates blind spots in supervision. There is no third-party supervisor in the testing vehicle, and the voice prompts and subtle physical movements of the examiners (such as touching the handbrake or lightly touching the gear shift) are difficult for supervisors outside the vehicle to detect. Traditional manual video surveillance relies on continuous monitoring by supervisors, which is limited by visual fatigue and insufficient police force, resulting in a very high rate of missed detection. Secondly, there is a lack of motivation for the management of socialized test centers. Currently, most test centers for driving test part 3 are managed by socialized entities. The test center operators lack the intrinsic motivation for self-supervision. Most of the examiners are recruited from outside the industry, and their professional qualities vary. Some of them take advantage of their positions to solicit "practice fees" or "guaranteed pass fees," and deliberately create a tense atmosphere for candidates who have not paid or directly step on the auxiliary brake to interfere with the test. Third, it is difficult to collect evidence of violations. The violations of the examiners in the vehicle (such as making hand gestures or touching the steering wheel) and the voice prompts (such as coughing hints or whispering instructions) are highly concealed. Traditional supervision methods can only record the entire process on video, and retrospective review requires a lot of manpower to check frame by frame, which cannot form timely and effective evidence for handling.

[0006] Existing technical solutions for driver's license examination supervision mostly focus on identifying examinee's operational behaviors (such as detecting line crossings and speeding), and have not yet addressed the comprehensive behavioral control of the accompanying examiner, a special subject. A few attempts to supervise accompanying personnel only involve single-action recognition (such as detecting secondary brake pedal application), lacking the ability to analyze the correlation between actions throughout the entire examination process, recognize speech and semantics, and fuse multi-source data. This makes it impossible to achieve closed-loop management from pre-examination registration and real-time warnings during the examination to post-examination traceability and analysis.

[0007] Therefore, an intelligent monitoring system that covers the entire life cycle of the examiner and integrates multimodal behavioral analysis is needed to solve the core problems of difficulty in monitoring, obtaining evidence, and handling cases involving examiners in the current driving test (subject 3). Summary of the Invention

[0008] In view of this, the present invention provides a method and system for intelligent monitoring of the behavior of driving test examiners in vehicle-mounted driving test (subject 3), in order to solve or alleviate one of the technical problems existing in the prior art, and at least provide a beneficial alternative.

[0009] The technical solution of this invention is implemented as follows: A method for intelligent monitoring of the behavior of driving test examiners in vehicle-mounted examinations (subject 3), comprising the following steps: The registration process for accompanying examiners is as follows: Collect the identity information, qualification certificates, and biometric data of all accompanying examiners for Subject 3 within the city, and establish a unified electronic file for accompanying examiners. The biometric data shall include at least facial images and voiceprint features. Pre-exam random vehicle assignment process: Before the start of each exam, based on a preset random algorithm, the registered accompanying examiners and examinees are randomly matched to generate a vehicle assignment list and synchronize it to the exam system; Identity verification steps: Before the test vehicle starts, the onboard facial recognition equipment will be used to perform liveness detection and facial comparison on the examinee to verify the consistency with the registered information. If the verification fails, the test will be prohibited from starting. Multimodal behavior monitoring steps during the exam: During the exam, multi-camera devices deployed in the vehicle collect panoramic video and close-up video of the auxiliary brake area, and audio data is collected in the vehicle through the onboard microphone; based on a pre-trained deep learning model, the video data is used to recognize the examiner's body behavior, and the audio data is used to recognize speech and non-speech voiceprints. The steps for determining violations are as follows: The identified physical behaviors are matched with the current test item to determine whether they belong to the set of violations under the corresponding test item; The identified voice content and non-voice voiceprints (such as coughing sounds) are compared with the preset violation voice template to determine whether they constitute a voice prompt violation; At the same time, the foot movements of the examiner in the vehicle are identified through video recognition of the secondary brake area to determine whether the secondary brake was applied. Real-time early warning steps: When a violation is detected, an early warning instruction containing the type of violation, time of occurrence, related examination items, and information of the personnel involved is immediately generated and pushed to the monitoring terminal. At the same time, video clips of preset duration before and after the violation are captured, marked, and stored. Post-exam source analysis steps: After the exam, summarize all behavioral data, warning records and exam results for each exam session to generate an on-board examiner behavior analysis report; based on the accumulation of data from multiple exam sessions, construct a behavioral profile of the on-board examiner, and statistically analyze the violation frequency, violation type distribution and related candidate pass rate indicators for each on-board examiner to provide a basis for regulatory decisions.

[0010] More preferably, the body behavior recognition specifically includes: For the simulated lighting test, identify whether the examiner in the vehicle is giving verbal instructions on lighting operation or coughing. For the starting test, identify whether the examiner in the vehicle touches the handbrake, gear shift, or steering wheel; For left and right turn items, identify whether the examiner in the vehicle verbally prompts the examiner to turn or assists with steering; For the straight-line driving test, identify whether the examiner in the vehicle is knocking on the car door or window; For lane-changing tests, identify whether the examiner in the vehicle coughs, uses hand gestures to indicate lane changes, or uses verbal or physical prompts to indicate the use of turn signals; For tests involving crossing pedestrian crossings, going straight through intersections, passing schools, and passing stations, the system identifies whether the examiner in the vehicle is applying the auxiliary brake. For the parallel parking test, identify whether the examiner in the vehicle is speaking or coughing to prompt the examiner to pull over, or making hand gestures. For the vehicle meeting test, identify whether the accompanying examiner is making hand gestures or tapping sounds; For overtaking maneuvers, identify whether the examiner in the vehicle uses verbal or physical prompts to signal the use of turn signals or hand gestures to signal the use of the horn. For U-turn tasks, identify whether the examiner in the vehicle uses verbal or physical prompts to activate the turn signal.

[0011] More preferably, the voice and non-voice voiceprint recognition specifically includes: After noise reduction processing of the collected audio data, the voice features of the examiner in the vehicle are extracted using voiceprint separation technology. The speech features are converted into text information and semantically matched with a pre-set violation warning word library corresponding to each test item. The violation warning word library contains keywords related to test operations, such as "turn on the light", "pull over", and "shift gears". At the same time, non-voice voiceprint features in the audio are identified. If continuous or specific rhythmic coughing or knocking sounds are detected and match the operation node time of the current test item, it is determined to be a violation of non-voice prompts.

[0012] More preferably, the identification of the secondary brake pedal application specifically includes: A wide-angle high-definition camera is deployed above the auxiliary brake pedal to capture real-time video of foot movements in the auxiliary brake area; The position of the examiner's right foot is located using a target detection model, and the foot's displacement trajectory is tracked. If the displacement of the foot pressing down on the secondary brake pedal exceeds a threshold within a preset time, and the vehicle is not in a situation requiring emergency avoidance, it is determined to be an illegal application of the secondary brake.

[0013] More preferably, the video data storage adopts a segmented storage strategy: The entire exam video is divided into independent time slices according to the exam items, and each slice is associated with the corresponding candidate information, accompanying examiner information, exam item identification, and violation mark; The in-vehicle panoramic video, the secondary brake close-up video, and the candidate information layer are combined to generate a composite video stream containing timestamps, exam items, and personnel identification. Only slices with violation markers and key nodes are retained during storage to reduce storage redundancy.

[0014] More preferably, the post-examination source tracing analysis step further includes: Construct a ranking model for violations by in-vehicle examiners, and dynamically rank all in-vehicle examiners in the city according to the number of violations and the severity of the violations; Construct a pass rate analysis model for in-vehicle examiners, statistically analyze the overall pass rate and the pass rate of each test item for examiners accompanying examiners, and identify in-vehicle examiners whose pass rates deviate abnormally. By cross-validating the ranking of violations with the pass rate analysis results, high-risk on-board examiners were identified and marked as key targets for supervision.

[0015] Further preferred options include: The registration management module is used to collect, store, and manage the identity information, qualification certificates, and biometric data of all driving test examiners in the city, so as to realize the unified registration and blacklist management of driving test examiners. The random vehicle allocation module interfaces with the examination system and generates a matching list of examiners and examinees based on a random algorithm to avoid human intervention in vehicle allocation; The identity verification module includes an in-vehicle face recognition unit, which is used to perform liveness detection and face comparison on the examinee in the vehicle before the exam and output the identity verification result. The multimodal data acquisition module includes an in-vehicle panoramic camera, a secondary brake close-up camera, and an in-vehicle microphone, which are used to acquire in-vehicle video, secondary brake area video, and in-vehicle audio data, respectively. The behavior analysis module includes a limb behavior recognition unit, a voice analysis unit, and a secondary brake pedal recognition unit, which are used to perform limb behavior recognition on video data, voice and non-voice voiceprint recognition on audio data, and secondary brake pedal action recognition on secondary brake area video, respectively. The early warning and handling module is used to receive the violation judgment results from the behavior analysis module, generate early warning information and push it to the monitoring terminal, and at the same time trigger the capture, marking and storage of violation video clips; The data analysis module is used to summarize in-exam behavioral data and post-exam results, generate behavioral profiles of on-board examiners, rankings of violations, and pass rate analysis reports, supporting regulatory personnel in conducting source tracing and decision-making.

[0016] In a further preferred embodiment, the data analysis module also includes a video compositing unit, used to synchronously composite the candidate information layer, the in-vehicle panoramic video stream, and the auxiliary brake close-up video stream in time to generate a composite video with an identity identifier and a timestamp; when the composite video is displayed on the monitoring terminal, it supports multi-screen split-viewing, simultaneously displaying candidate information, in-vehicle panoramic view, and auxiliary brake area view, making it easier for supervisors to intuitively judge violations.

[0017] Furthermore, the system also seamlessly integrates with third-party examination systems to obtain examination time, candidate information, examination progress, and examination result data in real time, and stores the behavioral analysis results in association with the examination data; The system allows regulators to perform combined queries based on multiple criteria, such as examinee's name, exam date, examiner's name, and type of violation, to retrieve corresponding composite videos, violation clips, and behavior analysis reports.

[0018] The embodiments of the present invention have the following advantages due to the adoption of the above technical solutions: First, a regulatory chain covering the entire lifecycle of the examination has been constructed, from registering examiners, randomly assigning vehicles before the exam, and verifying their identities, to real-time monitoring and early warning during the exam, and finally to post-exam traceability and analysis. By randomly assigning vehicles to prevent "favoritism," by verifying identities to prevent impersonation, and by using multimodal monitoring to capture hidden violations, the regulatory challenges of the closed vehicle environment have been fundamentally solved.

[0019] Second, by integrating multi-source data from video analysis (body movements, application of the auxiliary brake) and audio analysis (voice and non-voice voiceprints), a dedicated identification model has been established for 28 specific violations across 17 examination items. Compared to single-action recognition technology, the accuracy rate has been increased to over 96%, and the false alarm rate has been reduced to below 3%. For example, by using a close-up camera of the auxiliary brake and a foot movement tracking algorithm, it can accurately distinguish between normal foot placement and deliberate application of the auxiliary brake, avoiding the misjudgment problems of traditional pressure sensors.

[0020] Third, by adopting a segmented storage strategy based on examination items, only key video clips are retained, saving more than 60% of storage space compared to storing the entire recording. Through the synthesis of three images—examinee information, panoramic view inside the vehicle, and secondary brake view—supervisors can simultaneously obtain three types of information—personnel identity, overall environment, and key actions—on a single interface, significantly improving supervision efficiency.

[0021] Fourth, by constructing models such as on-board examiner behavior profiles, violation rankings, and pass rate correlation analysis, abnormal patterns that are difficult to detect by manual supervision, such as cross-regional mobile violations, concentrated violations of specific projects, and high violation-high pass rate correlations, can be uncovered.

[0022] Fifth, the system can send an alert to the monitoring terminal within 8 seconds of a violation occurring, and simultaneously store video evidence, enabling regulatory personnel to intervene promptly (such as through remote voice reminders or termination of the exam), thus overcoming the lag in traditional "post-event evidence collection."

[0023] Sixth, the system supports seamless integration with various third-party examination systems and can flexibly adapt to different examination management models in different regions; the registration management module supports unified standard management of in-vehicle examiners throughout the city and even the province, which facilitates the formulation of unified penalty standards and regulatory norms.

[0024] Seventh, traditional supervision methods require dedicated personnel to monitor surveillance videos, with each person supervising no more than five vehicles, and are prone to missing inspections due to visual fatigue. This system enables unattended automatic monitoring, allowing one supervisor to monitor more than 50 examination vehicles simultaneously, reducing police manpower consumption by 90%. Furthermore, by reducing examinee complaints and disputes, it indirectly lowers administrative mediation costs.

[0025] Eighth, by eliminating behaviors such as accompanying examiners assisting in cheating and demanding fees, a fair and just examination environment has been created, and examinee satisfaction has significantly improved.

[0026] The above overview is for illustrative purposes only and is not intended to be limiting in any way. In addition to the illustrative aspects, embodiments, and features described above, further aspects, embodiments, and features of the invention will become readily apparent from the accompanying drawings and the following detailed description. Attached Figure Description

[0027] To more clearly illustrate the technical solutions in the embodiments of this application or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0028] Figure 1 This is a schematic diagram of the system composition of the present invention; Figure 2 This is a schematic diagram illustrating the entire process management of the present invention; Figure 3 This is a simplified diagram of the system composition of the present invention; Figure 4 This is a flowchart of the application system of the present invention. Detailed Implementation

[0029] In the following description, only certain exemplary embodiments are briefly described. As those skilled in the art will recognize, the described embodiments can be modified in various ways without departing from the spirit or scope of the invention. Therefore, the drawings and description are considered to be exemplary in nature and not restrictive.

[0030] The embodiments of the present invention will now be described in detail with reference to the accompanying drawings.

[0031] Example 1: Daily supervision scenario of a driving test examiner accompanying a driver at a vehicle management office in a certain city.

[0032] This embodiment is applied to the monitoring of the driving test (Part 3) at the vehicle management office of a municipality. The city has 12 socialized testing centers, with approximately 800 examinees taking the test daily and 216 registered examiners. After system deployment, real-time monitoring of all Part 3 driving tests in the city will be achieved.

[0033] 1. System Deployment and Initial Configuration

[0034] First, electronic files were created for 216 examiners in vehicles through the filing management module. The information collected included: ID number, professional qualification certificate number, facial images (3 each of frontal, left, and right faces), and voiceprint samples (10-second audio recording of reading a preset text).

[0035] Three of the examiners who had previously been penalized for violations were placed on a key monitoring list, and higher thresholds were set for their behavioral recognition sensitivity.

[0036] Subsequently, the random vehicle allocation module was configured, and the allocation rules were set as follows: at 2:00 AM every day, a vehicle allocation list for that day is generated based on an improved chaotic algorithm to ensure that the matching probability between the same examiner and the same examinee is less than 0.1%. After the vehicle allocation list is generated, it is automatically encrypted and transmitted to the examination system of each examination room, and the examination room administrators have no right to modify it.

[0037] Multimodal data acquisition equipment is installed in each test vehicle: a 1080P wide-angle camera (120° field of view) is installed at the front of the roof, covering the passenger area and the front seat space; a 720P close-up camera (60° field of view) is installed directly above the passenger brake pedal to ensure unobstructed foot movements; and a high-sensitivity microphone array (48kHz sampling rate) is integrated into the rearview mirror, supporting 360° sound pickup. All equipment communicates in real time with the monitoring center server via an onboard terminal, with network latency controlled within 200ms.

[0038] 2. Examination Behavior Recognition and Early Warning Process

[0039] Taking the examination process of a candidate (admission ticket number: 202405001) as an example, the in-examination supervision process is described in detail: Pre-exam identity verification: At 08:30, the examiner, Zhang (registration number: TJ20230087), entered the test vehicle. The on-board facial recognition unit activated liveness detection, requiring Zhang to blink and turn his head. Simultaneously, facial images were collected and compared with the registration database. The similarity was 98.7%, and the verification was passed. The test system was then activated.

[0040] Simulated lighting test monitoring: At 08:32, the test entered the simulated lighting test. The voice analysis unit collected the in-vehicle audio in real time and recognized Zhang's voice command "turn on the low beams". It successfully matched the simulated lighting test violation prompt word library. At the same time, the behavior analysis unit detected that Zhang coughed twice in a row with an interval of 0.8 seconds, which is consistent with the characteristics of non-voice prompts.

[0041] The system immediately identified it as a "violation warning for the simulated lighting project" and generated an early warning instruction to be pushed to the monitoring terminal. The warning information included: "Time: 08:32:15, examiner: Zhang (TJ20230087), examinee: 202405001, examination item: simulated lighting, violation type: speaking to prompt lighting operation + coughing".

[0042] Simultaneously, a composite video segment (including candidate information, a panoramic view inside the vehicle, and the secondary brake screen) from 08:32:00 to 08:32:30 is extracted, marked, and stored.

[0043] Starting item monitoring: At 08:35, the test entered the starting stage. The body behavior recognition unit tracked Zhang's hand movements through the panoramic video inside the vehicle and detected that his right hand touched the handbrake 3 times and the gear shift 2 times in 2 seconds. This met the characteristics of the starting item violation. The system generated a "starting item violation touch" warning and stored the corresponding video clips at the same time.

[0044] Monitoring of the parallel parking exercise: At 08:52, the test entered the parallel parking phase. The behavior analysis unit detected that Zhang raised his right hand and waved his palm twice (approximately 30°) towards the right-side window, accompanied by a voice prompt saying "Park the side of the road". The system matched this action with the "Park the side of the road hand gesture" template and the voice prompt with the keyword "Park the side of the road", determining it to be a combined violation, issuing a warning and recording the video.

[0045] 3. Post-exam data analysis and processing

[0046] After the exam concluded that day, the data analysis module automatically generated the "May 20th In-Vehicle Examiner Behavior Analysis Report," with key data shown in the table below:

[0047] Based on the report, regulatory personnel interviewed Zhang and other individuals who frequently violated regulations, suspended their qualifications to accompany test takers, and conducted training for them; Zhao was listed as a key investigation target, and all test videos of him accompanying test takers in the past three months were retrieved for tracing and verification.

[0048] 4. System Operation Effectiveness

[0049] After the system had been running in the city for three months, statistics showed that the number of violations by examiners accompanying the test drivers decreased from 127 in the first month to 23 in the third month, a decrease of 81.9%. The number of complaints from test takers about "safety officers interfering with the exam" has decreased from 42 per month to 2; the average pass rate for the driving test (subject 3) in the city has stabilized at 68%-70% from 65%, with the fluctuation range narrowing, and the fairness of the exam has been significantly improved.

[0050] Example 2: Cross-regional joint inspection scenario

[0051] This embodiment simulates a joint inspection operation by the provincial traffic management department on the driving test sites in cities A and B under its jurisdiction, focusing on verifying the system's cross-domain data aggregation, analysis, and tracing capabilities.

[0052] 1. Cross-domain data connection and integration

[0053] The provincial regulatory platform connects with the existing driving test supervision systems in cities A and B through a standard interface to achieve real-time synchronization of three types of data: ① Basic data: Information on registered examiners in both cities (387 in total), examination venue information (8 examination venues in City A and 6 examination venues in City B), and examinee information; ② Data collected during the examination: real-time early warning information, synthesized video streams, and violation clips; ③ Post-exam data: Daily exam results and on-board examiner behavior analysis reports.

[0054] To ensure data security, cross-domain transmission is encrypted using the national cryptographic algorithm SM4, and the data transmission frequency is set to incremental synchronization every 5 minutes, with a full data verification performed once every morning.

[0055] 2. Discovering Abnormal Behavior Patterns

[0056] Investigators conducted a multi-dimensional correlation analysis of cross-domain data from the past month and discovered the following abnormal patterns: Cross-regional movement violation: Chen, the examiner accompanying the examinee (registered location: City A, number: AS2022017), appeared in three different test centers in City B within a month. The pass rate of the examinees he accompanied was as high as 92%, which is significantly higher than the average pass rate in City B (67%).

[0057] Further investigation revealed that Chen had two violations related to "pulling over and using hand signals" in City A, but no warning records were found during his exam in City B.

[0058] Violations concentrated in specific areas: In City A, violations related to applying the auxiliary brake during the "passing through school zones" test accounted for 72% of all violations of this type citywide, significantly higher than the percentages for other tests. Further analysis revealed that these violations were concentrated during the morning rush hour (07:30-08:30), involving 15 examiners, 8 of whom had multiple violation records.

[0059] Correlation between pass rate and violations: A scatter plot of “number of violations - pass rate” for examiners in the two cities was analyzed, and a significant negative correlation was found (correlation coefficient r=-0.83): examiners with ≥3 violations had an average pass rate of 89%; examiners with no violation records had an average pass rate of 63%.

[0060] 3. On-site verification and handling

[0061] In response to the above anomalies, the inspection team conducted an on-site investigation: A surprise inspection of Chen revealed a notebook among his belongings detailing the fees for "guaranteed pass" fees. It was confirmed that he had profited by subtly encouraging test-takers to pay fees in City B, thereby relaxing oversight. The inspection team immediately suspended his registration and transferred the case to the disciplinary inspection department for further processing.

[0062] In response to the high incidence of violations of applying the auxiliary brake when passing through school zones in City A, on-site investigation revealed that the traffic flow in this section of the road is high during the morning rush hour, and some examiners, in order to avoid responsibility, apply the auxiliary brake in advance to interfere with the examination.

[0063] The inspection team requested the vehicle management office in City A to optimize the examination schedule for that section of road, avoiding the morning rush hour, and to provide special training for the 15 examiners involved.

[0064] Those with "high violation rate and high pass rate" were interviewed, clearly informed that their behavior was suspected of cheating, ordered to write a written self-criticism, and included in the key screening dimensions of subsequent supervision.

[0065] 4. Statistics on audit results

[0066] This joint inspection identified 23 high-risk on-board examiners, verified 7 cases of assisting in cheating, and dismissed 5 unqualified on-board examiners.

[0067] One month after the inspection, the number of violations of applying the auxiliary brake while "passing through school zones" in City A decreased by 86%, and the difference in the average pass rate of accompanying examiners between the two cities narrowed from 25 percentage points to 4 percentage points, demonstrating a significant synergistic effect in cross-regional supervision.

[0068] Example 3: New Examination Room Acceptance and System Optimization Scenario

[0069] This embodiment describes the system deployment and acceptance process of a newly built socialized examination room (Examination Room C), as well as the optimization and iteration of the system model based on actual operating data.

[0070] 1. Deployment and functional testing of the new examination system

[0071] Test site C is planned to have 30 test vehicles, with an average daily testing capacity of 200 people.

[0072] The following tasks were completed during the system deployment phase: Hardware installation and debugging: One set of multimodal acquisition equipment was installed in each of the 30 vehicles. A server cluster (including 2 storage servers and 3 computing servers) was deployed in the examination site monitoring center to achieve 100% equipment online rate. Simulation tests verified the following: facial recognition accuracy rate 99.2%, voice recognition accuracy rate 97.8%, body movement recognition accuracy rate 96.5%, and auxiliary brake pedal operation recognition accuracy rate 99.5%.

[0073] Importing registration data: Import 42 examiners registered for test center C, and set the random vehicle assignment rule to "re-randomly match every half day" to avoid fixed partners.

[0074] Business process integration: Connect with the existing examination system in the examination room to verify the seamless integration of the entire process from candidate check-in, vehicle allocation, identity verification, examination process monitoring to result uploading, with 100% data consistency.

[0075] 2. Handling of abnormal data during trial operation

[0076] During the first week of system trial operation, the following abnormal data was collected, triggering the model optimization process: False alarm case 1: At 09:15 on March 5, the examiner Liu was identified by the system as "using hand gestures to prompt lane change" during the lane change test.

[0077] Reviewing the video revealed that Liu was adjusting the seatbelt position, and the natural swing of his arm was mistakenly identified as a gesture.

[0078] Optimization measures: Add a "gesture intent discrimination" sub-module to the body behavior recognition model, and make a comprehensive judgment by combining the arm swing speed (normal seat belt adjustment swing speed <15° / s, illegal gesture swing speed >30° / s), duration (normal action lasts <0.5s, illegal gesture lasts >1s), and head orientation (looking at the rearview mirror is normal operation, looking at the examinee is a violation warning).

[0079] Case 1 of omission: At 14:30 on March 7, Wu, the examiner accompanying the examinee, knocked on the car door to prompt the examinee to slow down during the straight driving test, but the system did not issue a warning.

[0080] Analysis of the cause: The noise inside the vehicle was too loud (about 75dB), and the microphone failed to clearly capture the knocking sound.

[0081] Optimization measures: Vibration sensor data fusion is added to the voice analysis module. The impact vibration (frequency range 50-200Hz) is detected by the accelerometer at the door and cross-validated with the audio data to improve the recognition rate of non-voice prompts.

[0082] Vehicle allocation rule optimization: During the trial operation, it was found that the waiting time for examinees was relatively long during the morning rush hour, and the original random vehicle allocation rule did not take into account the vehicle return time. Optimization measures: A "vehicle status factor" was added to the random vehicle allocation algorithm to prioritize matching examinees with vehicles that have already returned to the examination site, thus shortening the waiting time for examinees from an average of 22 minutes to 13 minutes.

[0083] 3. Functional verification during the acceptance phase

[0084] After the trial operation is completed, an expert acceptance team will be organized to conduct functional verification, focusing on testing the following scenarios: Full-process monitoring and verification: Five exams were randomly selected, and all stages from identity verification to post-exam analysis were tracked to confirm that the system functions met design requirements. The violation identification status for each exam item is shown in the table below:

[0085] Data analysis function verification: Retrieve summary data from the trial operation period, verify the completeness of the "On-board Examiner Behavior Analysis Report" generated by the system, and confirm that it includes the following content:

[0086] Stress test: Simulating a scenario with 200 test vehicles online simultaneously and 50 video streams per second, the system ran stably, with a peak CPU utilization of 78%, a peak memory usage of 65%, and a maximum video latency of 320ms, meeting the design requirements.

[0087] 4. System optimization results

[0088] After three months of operation at test center C, the optimized system has significantly improved key indicators: the false alarm rate has dropped from 12% during the trial run to 3.2%, and the missed alarm rate has dropped from 8% to 1.5%; the average waiting time for candidates has been shortened by 40%; the number of violations by examiners in the vehicle has decreased month by month, to 47, 29, and 16 respectively; and the candidate satisfaction survey shows that the "fairness of the examination" score has increased from 7.2 out of 10 during the trial run to 9.1.

[0089] The above description is merely a specific embodiment of the present invention, but the scope of protection of the present invention is not limited thereto. Any person skilled in the art can easily conceive of various variations or substitutions within the technical scope disclosed in the present invention, and these should all be included within the scope of protection of the present invention. Therefore, the scope of protection of the present invention should be determined by the scope of the claims.

Claims

1. A method for intelligent monitoring of the behavior of in-vehicle examiners in driving test subject 3, characterized in that, Includes the following steps: The registration process for accompanying examiners is as follows: Collect the identity information, qualification certificates, and biometric data of all accompanying examiners for Subject 3 within the city, and establish a unified electronic file for accompanying examiners. The biometric data shall include at least facial images and voiceprint features. Pre-exam random vehicle assignment process: Before the start of each exam, based on a preset random algorithm, the registered accompanying examiners and examinees are randomly matched to generate a vehicle assignment list; Identity verification steps: Before the test vehicle starts, the onboard facial recognition equipment will be used to perform liveness detection and facial comparison on the examinee to verify the consistency with the registered information. If the verification fails, the test will be prohibited from starting. Multimodal behavior monitoring steps during the exam: During the exam, multi-camera devices deployed in the vehicle collect panoramic video and close-up video of the auxiliary brake area, and audio data is collected in the vehicle through the vehicle microphone; based on a pre-trained deep learning model, the video data is used to recognize the examiner's body behavior, and the audio data is used to recognize speech and non-speech voiceprints. Steps for determining violations: The identified physical behaviors are matched with the current test item to determine whether they belong to the set of violations under the corresponding test item; The system compares the identified voice content and non-voice voiceprints (such as coughing sounds) with preset violation voice templates to determine whether a voice prompt violation has occurred; at the same time, it uses video recognition of the auxiliary brake area to identify the foot movements of the examiner in the vehicle to determine whether the auxiliary brake has been applied. Real-time early warning steps: When a violation is detected, an early warning instruction containing the type of violation, time of occurrence, related examination items, and information of the personnel involved is immediately generated and pushed to the monitoring terminal. At the same time, video clips of preset duration before and after the violation are captured, marked, and stored. Post-exam source analysis steps: After the exam, summarize all behavioral data, warning records and exam results for each exam session to generate an on-board examiner behavior analysis report; based on the accumulation of data from multiple exam sessions, construct a behavioral profile of the on-board examiner, and statistically analyze the violation frequency, violation type distribution and related candidate pass rate indicators for each on-board examiner to provide a basis for regulatory decisions.

2. The intelligent monitoring method for the behavior of the driving examiner in vehicle as described in claim 1, characterized in that, The body behavior recognition specifically includes: For the simulated lighting test, identify whether the examiner in the vehicle is giving verbal instructions on lighting operation or coughing. For the starting test, identify whether the examiner in the vehicle touches the handbrake, gear shift, or steering wheel; For left and right turn items, identify whether the examiner in the vehicle verbally prompts the examiner to turn or assists with steering; For the straight-line driving test, identify whether the examiner in the vehicle is knocking on the car door or window; For lane-changing tests, identify whether the examiner in the vehicle coughs, uses hand gestures to indicate lane changes, or uses verbal or physical prompts to indicate the use of turn signals; For tests involving crossing pedestrian crossings, going straight through intersections, passing schools, and passing stations, the system identifies whether the examiner in the vehicle is applying the auxiliary brake. For the parallel parking test, identify whether the examiner in the vehicle is speaking or coughing to prompt the examiner to pull over, or making hand gestures. For the vehicle meeting test, identify whether the accompanying examiner is making hand gestures or tapping sounds; For overtaking maneuvers, identify whether the examiner in the vehicle uses verbal or physical prompts to signal the use of turn signals or hand gestures to signal the use of the horn. For U-turn tasks, identify whether the examiner in the vehicle uses verbal or physical prompts to activate the turn signal.

3. The intelligent monitoring method for the behavior of the driving examiner in vehicle as described in claim 1, characterized in that, The speech and non-speech voiceprint recognition specifically includes: After noise reduction processing of the collected audio data, the voice features of the examiner in the vehicle are extracted using voiceprint separation technology. The speech features are converted into text information and semantically matched with a pre-set violation prompt word library corresponding to each test item. The violation prompt word library contains keywords related to test operations, such as "turn on the light", "pull over", and "shift gears". At the same time, non-voice voiceprint features in the audio are identified. If continuous or specific rhythmic coughing or knocking sounds are detected and match the operation node time of the current test item, it is determined to be a violation of non-voice prompts.

4. The intelligent monitoring method for the behavior of the driving examiner in vehicle as described in claim 1, characterized in that, The recognition of the auxiliary brake pedal application behavior specifically includes: A wide-angle high-definition camera is deployed above the auxiliary brake pedal to capture real-time video of foot movements in the auxiliary brake area; The position of the examiner's right foot is located using a target detection model, and the foot's displacement trajectory is tracked. If the displacement of the foot pressing down on the secondary brake pedal exceeds a threshold within a preset time, and the vehicle is not in a situation requiring emergency avoidance, it is determined to be an illegal application of the secondary brake.

5. The intelligent monitoring method for the behavior of in-vehicle examiners in Subject 3 according to claim 1, characterized in that, The video data storage adopts a segmented storage strategy: The entire exam video is divided into independent time slices according to the exam items, and each slice is associated with the corresponding candidate information, accompanying examiner information, exam item identification, and violation mark; The in-vehicle panoramic video, the secondary brake close-up video, and the candidate information layer are combined to generate a composite video stream containing timestamps, exam items, and personnel identification. Only slices with violation markers and key nodes are retained during storage to reduce storage redundancy.

6. The intelligent monitoring method for the behavior of the driving examiner in vehicle as described in claim 1, characterized in that, The post-exam source tracing analysis steps also include: Construct a ranking model for violations by in-vehicle examiners, and dynamically rank all in-vehicle examiners in the city according to the number of violations and the severity of the violations; Construct a pass rate analysis model for in-vehicle examiners, statistically analyze the overall pass rate and the pass rate of each test item for examiners accompanying examiners, and identify in-vehicle examiners whose pass rates deviate abnormally. By cross-validating the ranking of violations with the pass rate analysis results, high-risk on-board examiners were identified and marked as key targets for supervision.

7. A smart monitoring system for the behavior of in-vehicle examiners in driving test subject 3, characterized in that, include: The registration management module is used to collect, store, and manage the identity information, qualification certificates, and biometric data of all driving test examiners in the city, so as to realize the unified registration and blacklist management of driving test examiners. The random vehicle allocation module interfaces with the examination system and generates a matching list of examiners and examinees based on a random algorithm to avoid human intervention in vehicle allocation; The identity verification module includes an in-vehicle face recognition unit, which is used to perform liveness detection and face comparison on the examinee in the vehicle before the exam and output the identity verification result. The multimodal data acquisition module includes an in-vehicle panoramic camera, a secondary brake close-up camera, and an in-vehicle microphone, which are used to acquire in-vehicle video, secondary brake area video, and in-vehicle audio data, respectively. The behavior analysis module includes a limb behavior recognition unit, a voice analysis unit, and a secondary brake pedal recognition unit, which are used to perform limb behavior recognition on video data, voice and non-voice voiceprint recognition on audio data, and secondary brake pedal action recognition on secondary brake area video, respectively. The early warning and handling module is used to receive the violation judgment results from the behavior analysis module, generate early warning information and push it to the monitoring terminal, and at the same time trigger the capture, marking and storage of violation video clips; The data analysis module is used to summarize in-exam behavioral data and post-exam results, generate behavioral profiles of on-board examiners, rankings of violations, and pass rate analysis reports, supporting regulatory personnel in conducting source tracing and decision-making.

8. The intelligent monitoring system for the behavior of the driving examiner in vehicle as described in claim 7, characterized in that, The data analysis module also includes a video compositing unit, which is used to time-synchronize and synthesize the candidate information layer, the in-vehicle panoramic video stream, and the auxiliary brake close-up video stream to generate a composite video with an identity identifier and a timestamp. When the composite video is displayed on the monitoring terminal, it supports multi-screen split-viewing, simultaneously displaying candidate information, in-vehicle panoramic view, and auxiliary brake area view, making it easier for supervisors to intuitively judge violations.

9. The intelligent monitoring system for the behavior of the driving examiner in vehicle as described in claim 7, characterized in that, The system also seamlessly integrates with third-party examination systems to obtain examination time, candidate information, examination progress, and examination result data in real time, and stores the behavioral analysis results in association with the examination data. The system allows regulators to perform combined queries based on multiple criteria, such as examinee's name, exam date, examiner's name, and type of violation, to retrieve corresponding composite videos, violation clips, and behavior analysis reports.

Images