Test and evaluation method for driver monitoring system based on simulation robot

Abstract

The invention relates to a driver monitoring system test evaluation method based on a simulation robot. Comprising the following steps: step 1, constructing a simulation robot for a DMS test; 2, designing a DMS test simulation robot control system; step 3, establishing a DMS system; step 4, generating and evaluating a test case; according to the method, firstly, a simulation robot used for DMS testing is constructed, then a steering engine is selected and arranged, and fine control over fatigue actions of the simulation robot is achieved through control of a microcontroller; a python language is used as a development basis, modular design is adopted, eye detection, head posture estimation, attention evaluation, parameter configuration, a camera interface and the like are included, and a detected DMS system is deployed; and finally, based on the national standard and the actual driving behavior, generating an efficient and high-coverage test case set by adopting a combined test theory, and executing through a control system, so that the problems of low efficiency, high cost, uncontrollable action, insufficient scene coverage, lack of a standardized evaluation system and the like of the existing DMS test method are solved.

Description

TECHNICAL FIELD

[0001] The application belongs to the technical field of intelligent vehicles, and particularly relates to a driver monitoring system test evaluation method based on a simulation robot. BACKGROUND

[0002] At present, as a key active safety function in an intelligent cockpit, a driver monitoring system needs to undergo a large amount of tests in real or simulated environments before mass production. However, the traditional real person test method has the disadvantages of low efficiency, difficulty in reproduction, high risk, high cost and the like; and a static dummy cannot completely restore the fatigue behavior characteristics due to the lack of real actions, and cannot accurately reflect the detection capability of the DMS system on dynamic fatigue behavior. Although some simulation robots can realize basic actions, they still stay in the stage of structural construction and lack systematic control strategies and standardized test processes, and cannot support complex behavior tests; meanwhile, there is a lack of a complete set of standardized and automated test process framework in the existing DMS test, and the recognition capability of the system on composite fatigue actions cannot be accurately evaluated. SUMMARY

[0003] To solve the above problems, the application provides a driver monitoring system test evaluation method based on a simulation robot, which first constructs a simulation robot for DMS test, then selects and arranges a steering engine, and realizes fine control of fatigue actions of the simulation robot through a microcontroller, takes python language as the development basis, adopts modular design, includes eye detection, head posture estimation, attention evaluation, parameter configuration, camera interface and the like, and deploys a DMS system under test; finally, based on national standards and actual driving behavior, a high-efficiency and high-coverage test case set is generated by using a combination test theory, and is executed through a control system, a complete test process is constructed, and the problems of low efficiency, high cost, uncontrollable actions, insufficient scene coverage and lack of a standardized evaluation system in the existing DMS test method are solved.

[0004] The technical scheme of the application is described below in combination with the drawings:

[0005] The application provides a driver monitoring system test evaluation method based on a simulation robot, which includes the following steps:

[0006] Step one, building a simulation robot for DMS test;

[0007] The manufacturing of parts is completed through 3D printing technology, and the production of facial simulation skin is completed by using a liquid silicone injection molding process;

[0008] Step two, designing a DMS test simulation robot control system;

[0009] Different types of steering engines are selected; an I2C bus is connected between the Arduino UNO and the PCA9685; the PCA9685 is used as a PWM output device to control the steering engine; the steering engine is controlled through the PWM output device; the steering engine is controlled through the PWM output device;2 C control scheme; through command analysis and task scheduling mechanism, realize the fine control of multiple rudder channel, drive the simulation robot to complete the standardization face and head behavior;

[0010] Step three, build DMS system;

[0011] Build a driver monitoring system based on visual perception, the driver monitoring system uses camera to collect video stream in real time, monitors the face state of the simulation robot, extracts the parameters of eye closing degree, yawning frequency and head posture, and compares them with the set threshold value, to realize the dynamic identification and grading judgment of the driver fatigue and distraction state;

[0012] Among them, the DMS system includes face and eye key point detection module, eye movement detection module, head posture estimation module and attention evaluation module;

[0013] The face and eye key point detection module is used to extract and track the key point coordinates in the face area, supporting the subsequent action recognition and posture estimation;

[0014] The eye movement detection module is used to calculate eye parameters based on key points to judge fatigue state;

[0015] The head posture estimation module is used to estimate the three-dimensional posture angle of the head according to the key points to judge the distraction state;

[0016] The attention evaluation module is used to comprehensively analyze the eye movement, head posture and other information, and output the attention dispersion state or warning of the driver;

[0017] Step four, generate and evaluate test cases;

[0018] Based on the combination test theory and greedy algorithm, design standardized test cases covering different fatigue actions, including single behavior and multi behavior combination scene, and control instructions are issued through serial port to drive the simulation robot to execute specific action sequence in real time, and compare the recognition results of DMS system with known action label to calculate performance index.

[0019] Further, the specific method of step one is as follows:

[0020] 11) Structure design and 3D printing: design head mechanical parts including eye, mouth, forehead and neck, and use 3D printing technology to manufacture each part;

[0021] 12) Part assembly: assemble each part after printing to form a complete head skeleton structure, and polish and lubricate the piston assembly;

[0022] 13) Silicone face making and installation: After polishing and polishing, the resin 3D printing mold is sprayed with a release agent, and then liquid silicone injection is performed to make a simulated human face skin; the silicone face is firmly installed on the robot face structure by adhesive method.

[0023] Further, the specific method of step two is as follows:

[0024] 21) Select and arrange different steering gears according to the required torsion angle and torsion torque of different structural parts.

[0025] 22) The system supports simultaneous scheduling and execution of up to 16 steering gear tasks, each task containing steering gear channel number, starting angle, target angle, holding time, forward time and return time; The adjustment of steering gear speed is based on time interpolation algorithm, which divides each steering gear action into several small steps, gradually approaches the target angle in each time slice, simulates natural human action; The system supports fixed time after completing the action, and supports automatic return to the initial angle to realize the closed loop of the action; The user inputs the instruction in a specific format through the serial port, and the system automatically parses and generates the task list; Use the angleToMicroseconds() function to map the angle to the standard PWM pulse width, and output 4096 resolution high precision PWM signal through PCA9685 module to realize precise control of steering gear angle.

[0026] Further, the specific method of step 21) is as follows:

[0027] JX PDI-1109MG: for small torque;

[0028] JX PDI-6221MG: for medium torque such as head rotation, lower jaw opening and closing;

[0029] HS-805BB: for heavy load.

[0030] Further, the specific method of step three is as follows:

[0031] 31) Image acquisition and preprocessing: OpenCV is called to obtain video stream through camera, and grayscale conversion, channel reconstruction and bilateral filter denoising are performed;

[0032] 32) Integrate MediaPipe Face Mesh model to detect 468 facial key points in real time; First, the model inference output is normalized 3D coordinates; 3D coordinate sequence is represented in normalized space defined by formula (1):

[0033]

[0034] In the formula, w, h are the width and height of the input image; d refis the reference depth value based on the face proportion; the normalized coordinates are converted into absolute positions in the image pixel coordinate system through linear mapping of formula (2): (x norm y norm , z rel is the coordinate of the original face key point in the 3D coordinate system; x and y are the pixel positions of the key point on the image plane; z is the depth value of the point relative to a certain reference plane, which is the relative depth obtained by model inference and is used to depict the stereo geometric features of the face structure;

[0035]

[0036] In the formula, (x pixel , y pixel ) are image pixel coordinates after the normalized coordinates are restored to the original image size through linear mapping;

[0037] All face key points are enumerated and screened, including eye key points, iris key points and pose estimation key points. After screening the key points, the system visualizes and labels the eye-related key points in the image to assist in debugging and result verification. At the same time, the head pose estimation module estimates the head pose based on the selected face key points, and draws three-dimensional coordinate axes in the image to intuitively show the orientation and angle change of the head;

[0038] 33) After completing the face key point extraction and coordinate optimization, the system expands the calculation of the multi-modal features of eye movement, gaze direction and head pose based on the key point sequence; according to the selected 6 key points of the left and right eyes, the aspect ratio of the eyes is calculated, and the aspect ratio ratio is calculated through formula (3) to obtain the Euclidean distance relationship between the key points in the vertical and horizontal directions of the eyes:

[0039]

[0040] In the formula, P1, P2, … P6 are the coordinates of the left and right eye key points, which are used to calculate the aspect ratio of the eyes;

[0041] After obtaining the EAR value of the left and right single eyes, the average value of the EAR values of the left and right single eyes is calculated. If the average value is less than 0.2, it is marked as a closed eye state, the proportion of closed eye time in the unit time window PERCLOS is counted, and the driver fatigue degree is judged through the proportion PERCLOS. If the proportion PERCLOS exceeds the set threshold value, the system will trigger a warning;

[0042] The iris center points of the left and right eyes are located, and a local eye coordinate system is established in combination with the corner key points; the relative position of the iris in the white eye area is estimated by comparing the offset of the iris center relative to the corner, and then the line-of-sight direction vector is derived; the line-of-sight direction vector describes the user's current gaze direction as being straight ahead, left-biased, right-biased, up-biased, or down-biased;

[0043] The system estimates the three-dimensional pose angle of the head through a key point-based geometric modeling method;

[0044] 34) The driver monitoring function is disassembled into multiple independent functional modules, including a face and eye key point detection module, an eye movement detection module, a head pose estimation module, and an attention evaluation module;

[0045] The face and eye key point detection module is used to extract and track the key point coordinates in the face region, supporting subsequent action recognition and pose estimation;

[0046] The eye movement detection module is used to calculate eye parameters based on key points to determine the fatigue state;

[0047] The head pose estimation module is used to estimate the three-dimensional pose angle of the head based on key points to determine the distraction state;

[0048] The attention evaluation module is used to comprehensively analyze eye movement and head pose information and output the driver's attention distraction state or warning.

[0049] Further, the specific method of the system estimating the three-dimensional pose angle of the head through a key point-based geometric modeling method in step 33) is as follows:

[0050] The system selects face key points that are stable in position, not easily affected by expression changes, and have consistent structure semantics from the 468 face key points extracted by MediaPipe FaceMesh as reference points; after the key points are extracted, the head pose estimation module estimates the three-dimensional pose angle of the head based on the PnP method; specifically, first, the known face three-dimensional model points are matched with the two-dimensional projection coordinates in the image, and the internal parameters of the camera are combined to call the cv2.solvePnP() function provided by OpenCV to solve the rotation vector R and the translation vector t; the three-dimensional-two-dimensional point matching is realized by minimizing the re-projection error problem of formula (4), and then the pose information of the head in the three-dimensional space is solved;

[0051]

[0052] In the formula, π(·) is the camera projection function; K is the internal parameter matrix (including focal length f, principal point (cx, cy)); X i is the three-dimensional face model key point coordinate; xi For the corresponding two-dimensional pixel coordinates; the rotation vector is then converted to a rotation matrix via cv2.Rodrigues(), and further decomposed to obtain three Euler angle parameters of the head, which are calculated by formula (5):

[0053]

[0054] In the formula, r ij is the i-th row and j-th column element of the rotation matrix R; arctan2(y, x) is the four quadrant inverse tangent function, the output range is (-π, π], through Convert radians to degrees; the system sets an offset threshold for each angle to determine whether the driver has a non-concentrated behavior; when any attitude angle continuously exceeds the set range and exceeds the time threshold, the AttentionScorer module will output the attention distraction state, triggering the corresponding prompt or warning mechanism.

[0055] Further, the specific method of step four is as follows:

[0056] 41) Design a test action combination containing multi-dimensional actions such as eye opening and closing, mouth opening and closing, head twisting and pitching, and build a systematic test case set; the test scheme sets a duration gradient of 0.5s, 1s difference;

[0057] The multi-time dimension test scheme is based on the complexity of actual driving scenarios: short eye closure, i.e. <0.5s, is within the normal blinking range, while continuous eye closure, i.e. ≥2s, is considered as a potential sign of fatigue driving; looking down for 1 second is a short gaze at the instrument or vehicle display screen, and if the duration exceeds 3 seconds, there is a risk of driving;

[0058] For eye closure duration, 2s is selected as the center point, and 0.5s and 1s are selected as the difference time interval, with 0.5, 1, 2, 2.5, and 3 as the test parameter values; for yawning duration, 0.5, 1, 3, 3.5, and 4 are selected as the parameters; the parameter values for head posture abnormality testing are set according to the definition of head posture abnormality in the national standard;

[0059] 42) Introduce multi-element combination testing, focusing on the interaction of three core behavior dimensions: ① eye closure, ② mouth opening, and ③ head posture deviation; if the three behaviors appear simultaneously or sequentially, it indicates an abnormal driving state;

[0060] 43) The data used in the combined test theoretical model are all discrete data and there is no constraint between them; in the test process, the test intensity k represents the coverage degree of the test, and a double-factor combination test case is designed with test intensity k=2; under the condition of k=2, a greedy algorithm is used to cover all two-parameter combinations;

[0061] 44) According to the test case, the test is carried out, including closing eyes, abnormal head posture, yawning; all tests are carried out in a controlled variable consistent environment, real-time image acquisition is carried out by using a camera, relevant actions are simulated by using a simulation robot, the detection rate, false alarm rate, average of frame per second and average of single frame processing time of the DMS system are counted, so as to evaluate the DMS system.

[0062] Further, the logic of the greedy algorithm in step 43) is as follows:

[0063] 431) Initialize data: set the cover set to be empty set, and all two two combinations are covered;

[0064] 432) In the candidate test case, select the one that can cover the most two two combinations that have not appeared, and add it to the final test case set;

[0065] 433) Mark the two two combinations contained in the test case as covered;

[0066] 434) Repeat steps 422) and 423) until all two two combinations are covered at least once.

[0067] The beneficial effects of the present application are:

[0068] 1) The current test specification of the driver fatigue monitoring system is still not comprehensive, and the real person test is relatively tedious and time-consuming, the test based on the simulation robot can not only effectively improve the test efficiency, but also the test precision is easier to control, the random error of the real action is eliminated, the extreme scene and the composite scene can be covered in the test process, the safety hidden danger is avoided, and the test has high controllability and repeatability, which is helpful to accurately evaluate the system performance;

[0069] 2) The application constructs a simulation robot head, prints and splices parts, controls the simulation robot head through a rudder, generates corresponding actions, finally builds a driver monitoring system, tests the state of the driver in real time through a camera, finally tests the driver monitoring system, and measures the DMS through the detection rate, false alarm rate, frame per second and single frame processing time, provides strong data support and optimization direction for the optimization of the DMS system. BRIEF DESCRIPTION OF DRAWINGS

[0070] In order to more clearly illustrate the technical scheme of the embodiments of the present application, the following will briefly introduce the drawings needed to be used in the embodiments, and it should be understood that the following drawings only show some embodiments of the present application, and should not be regarded as limiting the scope, and for those skilled in the art, other related drawings can also be obtained without creative labor on the basis of these drawings.

[0071] Figure 1 for the flow chart of the present application;

[0072] Figure 2 for the simulation robot real object effect drawing;

[0073] Figure 3 for the simulation robot control system circuit layout;

[0074] Figure 4 for the DMS system face key point enumeration diagram;

[0075] Figure 5 for the eye feature point schematic diagram. DETAILED DESCRIPTION

[0076] The application will be further described below in conjunction with the drawings and embodiments. It can be understood that the specific embodiments described herein are only used to explain the application, and not to limit the application. In addition, it should be noted that, for the convenience of description, only the parts related to the application are shown in the drawings, not all the structures.

[0077] Referring to Figure 1 The embodiment provides a driver monitoring system test evaluation method based on a simulation robot, and comprises the following steps:

[0078] Step 1: Build a simulation robot for DMS test;

[0079] The manufacturing of parts is completed through 3D printing technology, and the manufacturing of facial simulation skin is completed through liquid silicone injection molding process, and the specific steps are as follows:

[0080] 11) Structure design and 3D printing: design the head mechanical parts containing eye, mouth, forehead, neck and other multi-degree-of-freedom motion structures, and manufacture each part by using 3D printing technology (such as FDM or SLA). The printing materials are preferably PLA (for shell and decoration) and ABS (for force structure parts such as rudder frame and connecting rod support), and the printing process needs to optimize the support structure design and cooling strategy to prevent warping deformation;

[0081] 12) Part assembly: high-precision assembly of the printed parts is performed to form a complete head skeleton structure, and the piston assembly is polished and lubricated to ensure smooth movement;

[0082] 13) Silicone face making and installation: high-precision resin 3D printing mold is used, which is finely polished and sprayed with release agent, then liquid silicone injection is carried out, and after demolding and cutting, a simulated human face skin with real skin feel and elasticity is made. The silicone face is firmly installed on the robot face structure by means of magnet, magic tape, double-sided tape and local silicone adhesion, which is convenient for subsequent generation of stable, fine and more realistic facial expressions. The final simulation robot physical effect diagram is shown in Figure 2 ;

[0083] Step two, design DMS test simulation robot control system;

[0084] Select different types of servo motors; adopt I 2 C control scheme based on Arduino UNO and PCA9685; through command analysis and task scheduling mechanism, fine control of multiple servo channels is realized, and the simulation robot completes standardized facial and head behavior, as follows:

[0085] 21) According to the required torsion angle and torsion torque of different structural parts, different servo motors are selected and arranged.

[0086] JX PDI-1109MG: used for small torque parts such as eyelids and eyebrows (2.2kg·cm under 4.8V working voltage, maximum torsion angle 120°);

[0087] JX PDI-6221MG: used for head rotation, jaw opening and closing, etc. (17.25kg·cm under 4.8V working voltage, maximum torsion angle 120°);

[0088] HS-805BB: used for heavy load such as head pitching (19.8kg·cm under 4.8V working voltage, maximum torsion angle 120°);

[0089] 22) Based on Arduino UNO microcontroller and PCA9685 PWM expansion module, multiple servo motors are controlled through I 2 C communication, realizing functions such as multi-channel parallel control, motion smooth interpolation, motion retention and automatic return; the circuit arrangement is shown in Figure 3, through the command analysis and task scheduling mechanism, the fine control of multiple servo channels is realized, and the simulation robot is driven to complete the standardized face and head behavior action. The system supports up to 16 servo tasks to be executed simultaneously, and each task contains parameters such as servo channel number, starting angle, target angle, holding time, forward time and return time. The adjustment of the speed of the servo is based on the time interpolation algorithm, which divides each servo action into several small steps, and gradually approaches the target angle in each time slice, ensuring smooth and jitter-free servo motion, simulating natural human motion. The system supports keeping a fixed time after completing the action (for simulating continuous behaviors such as closing eyes and opening mouth), and supports automatically returning to the initial angle to realize the closed loop of the action. Users can input specific format instructions through the serial port (such as: multi 0 90 150 1.0 1.5 1.5|1 60 120 1.5 2.0 2.0), and the system automatically parses and generates a task list, which is convenient for linking other test platforms or PC software to realize unified scheduling. The angleToMicroseconds() function is used to map the angle to the standard PWM pulse width (600-2400us), and the PCA9685 module outputs a high-precision PWM signal with 4096 resolution, realizing precise control of the servo angle.

[0090] Step three, build a DMS system;

[0091] A driver monitoring system based on visual perception is built, which uses a camera to collect video streams in real time, monitors the face state of the simulation robot, extracts the parameters of the eye closure degree, yawning frequency and head posture, and compares them with the set threshold to realize dynamic recognition and grading judgment of the driver's fatigue and distraction state;

[0092] The DMS system includes a face and eye key point detection module, an eye movement detection module, a head posture estimation module, and an attention evaluation module.

[0093] The face and eye key point detection module is used to extract and track the key point coordinates in the face region, supporting subsequent action recognition and posture estimation.

[0094] The eye movement detection module is used to calculate eye parameters based on key points to determine the fatigue state.

[0095] The head posture estimation module is used to estimate the three-dimensional posture angle of the head based on the key points to determine the distraction state.

[0096] The attention evaluation module is used to comprehensively analyze eye movement, head posture and other information, and output the driver's attention dispersion state or warning, as follows:

[0097] 31) Image acquisition and preprocessing, the camera is called by OpenCV to obtain video stream, and the grayscale conversion, channel reconstruction (single channel → pseudo three channels) and bilateral filter denoising are performed to eliminate the light interference while retaining the facial edge features.

[0098] 32) Real-time detection of 468 facial key points (including 10 iris points) by integrating MediaPipe Face Mesh model, see Figure 4 First, the model inference output is normalized 3D coordinates; the coordinate sequence is represented in the normalized space defined by equation (1):

[0099]

[0100] where w, h are the input image width and height, d ref is the reference depth value based on the face ratio. Through the linear mapping of formula (2), the normalized coordinates are converted into absolute positions in the image pixel coordinate system:

[0101]

[0102] This mapping method preserves the z rel relative depth information, and the boundary constraints such as x pixel ∈[0,w-1] ensure the validity of the coordinates. Then, all facial key points are enumerated and screened, including eye key points, iris key points and pose estimation key points. After screening the key points, the system first visualizes the eye-related key points in the image through the EyeDetector.show_eye_keypoints() method to assist debugging and result verification; At the same time, the HeadPoseEstimator.get_pose() module estimates the head pose based on the selected facial key points, and draws a three-dimensional coordinate axis (XYZ) in the image to intuitively show the orientation and angle change of the head, enhancing the visual interaction experience.

[0103] 33) After completing the face key point extraction and coordinate optimization, the system calculates the multi-modal features such as eye movement, gaze direction and head pose based on the key point sequence, so as to realize the evaluation of the driver's fatigue state. According to the selected 6 key points of the left and right eyes, see Figure 5 , the aspect ratio of the eyes is calculated, which is obtained by calculating the Euclidean distance relationship between the key points in the vertical and horizontal directions of the eyes by formula (3):

[0104]

[0105] After obtaining the EAR values of the left and right monocular eyes, the average value of the two is calculated. If the average value is less than 0.2, it is marked as a closed eye state, and the PERCLOS of the closed eye time in the unit time window is calculated. Through this index, the degree of driver fatigue is judged. If the proportion exceeds the set threshold, the system will trigger a warning;

[0106] To estimate the direction of implementation, the iris center points of the left and right eyes are located, and a local eye coordinate system is established in combination with the key points of the eye corners. By comparing the offset of the iris center relative to the eye corner, the relative position of the iris in the white eye area can be estimated, and the gaze vector is derived. This vector can describe the current gaze direction of the user as straight ahead, left, right, up or down.

[0107] To accurately analyze the attention state of the driver, the system estimates the three-dimensional pose angle (Pitch, Yaw, Roll) of the head through a key point-based geometric modeling method. In specific implementation, the system selects position points with stable geometric features (such as the tip of the nose, the corners of the eyes, the corners of the mouth, and the chin) from the 468 facial key points extracted by MediaPipe FaceMesh as reference points. After the key points are extracted, the HeadPoseEstimator module calls the get_pose() method to estimate the three-dimensional pose angle (Pitch, Yaw, Roll) of the head using the PnP (Perspective-n-Point) method to reflect the changes in the driver's orientation. Specifically, this method first corresponds the known three-dimensional model points of the face (such as the tip of the nose, the corners of the eyes, the corners of the mouth, and the chin) with the two-dimensional projection coordinates in the image, and combines the camera's intrinsic parameters (such as focal length, principal point, distortion parameters) to call the cv2.solvePnP() function provided by OpenCV to solve the rotation vector R and the translation vector t. The mathematical essence is to realize the three-dimensional-two-dimensional point registration through the minimization of the re-projection error problem in equation (4), and then solve the head's pose information in three-dimensional space.

[0108]

[0109] In the formula, π(·) is the camera projection function, K is the intrinsic matrix (including focal length f, principal point (cx, cy)), X i is the three-dimensional face model key point coordinate, x i is the corresponding two-dimensional pixel coordinate. The rotation vector is then converted to a rotation matrix via cv2.Rodrigues(), and further decomposed to obtain the three Euler angle parameters of the head. The Euler angles are calculated by equation (5):

[0110]

[0111] Where r ij : The element in the i-th row and j-th column of the rotation matrix R; arctan2(y,x): the arctangent function in the four quadrants, with an output range of (-π,π]. Convert radians to angles. The system sets offset thresholds for each angle (e.g., pitch_thresh, yaw_thresh, roll_thresh) to determine if the driver is exhibiting inattentive behavior: such as frequently looking down at a phone (Pitch), frequently turning the head to talk to passengers (Yaw), or abnormal head movements (Roll). When any posture angle continuously exceeds the set range and surpasses the time threshold (pose_time_thresh), the AttentionScorer module will output a "Distracted" status, triggering corresponding prompts or warning mechanisms.

[0112] 34) The complex driver monitoring function is broken down into multiple functional modules with clear responsibilities and independent functions: face and eye key point detection module, eye movement detection module, head posture estimation module and attention assessment module, etc., which facilitates the maintenance and expansion of the system, as well as the testing, debugging and optimization of subsequent functions.

[0113] The facial eye key point detection module is used to extract and track the coordinates of key points in the facial area to support subsequent action recognition and pose estimation.

[0114] The eye-tracking detection module is used to calculate the aspect ratio of the eyes, identify the closed-eye state, and the direction of gaze.

[0115] The head pose estimation module is used to estimate the Euler angles of the head and determine the orientation change;

[0116] The attention assessment module is used to comprehensively analyze eye movement and head posture information, and output the driver's attention distraction status (if the blinking time exceeds the threshold, it will display "tired"; if the driver is off-line, it will display "distracted") or warning.

[0117] Step 4: Generate and evaluate test cases;

[0118] Based on combinatorial testing theory and greedy algorithms, standardized test cases covering different fatigue actions are designed, including single-behavior and multi-behavior combination scenarios. Control commands are issued via serial port to drive the simulated robot to execute specific action sequences in real time. Performance indicators are calculated by comparing the DMS system recognition results with known action labels, as detailed below:

[0119] 41)Based on the accurate definition of behavior and the prompt conditions issued in GB / T 41797-2022 standard, and in view of the detection characteristics of DMS system and the actual application requirements, the application scientifically designs a test action combination containing eye opening and closing, mouth opening and closing, head twisting and pitching and other multi-dimensional actions, and constructs a systematic test case set. The test scheme specially sets the duration gradient of 0.5s, 1s and other difference values. Setting multiple time dimension tests is mainly based on the complexity of the actual driving scene: short eye closure (<0.5s) may belong to the category of normal blinking, while continuous eye closure (≥2s) can be regarded as a potential sign of fatigue driving; Similarly, lowering the head for 1 second may be a short gaze on the instrument or vehicle display screen, and if the duration exceeds 3 seconds, there is a high risk of driving. By introducing a hierarchical design of different time dimensions, the test scheme can effectively test the recognition sensitivity and judgment accuracy of the system for various key actions under different durations, thereby providing real data basis for the accuracy calibration and complex function improvement of the DMS system.

[0120] Table 1 sets the test scene parameter categories and value ranges. For eye closure duration, 2s is selected as the center point according to the national standard GB / T41797-2022, and the difference time interval of 0.5s and 1s, 0.5, 1, 2, 2.5, 3 five test parameter values are selected, which not only can effectively distinguish between blinking and abnormal eye closure, but also can comprehensively cover different risk levels of eye closure behavior in driving scenes. For the yawning duration parameter, 0.5, 1, 3, 3.5, 4 are designed, which not only cover the test range of the national standard, effectively distinguish between light mouth movement and yawning, and help to evaluate the recognition boundary and false alarm probability of the DMS system for yawning. The parameter values of the head posture abnormality test are mainly set according to the definition of head posture abnormality in the national standard, which also covers the test range of the national standard, distinguishes between transient natural action and abnormal posture, and covers more abnormal behaviors of different risk levels.

[0121] Table 1 Test scene parameter categories and their values

[0122]

[0123]

[0124] 42)In the performance verification of the driver monitoring system, considering that multiple fatigue or distraction features (such as eye closure, mouth opening, head deflection, etc.) may occur simultaneously during actual driving, in order to improve the practicality and robustness of the system, single-element testing cannot fully cover the recognition ability of the system. Therefore, we further introduce multi-element combination testing, especially focusing on the interaction of three core behavior dimensions: ① eye closure (reflecting fatigue / drowsiness), ② mouth opening (reflecting yawning / communication), ③ head posture deviation (reflecting distraction / attention shift). The simultaneous or sequential occurrence of these three types of behavior is likely to indicate abnormal driving state. Given the large number of parameter dimensions involved in the test, using an exhaustive test scheme would result in an exponential increase in the number of test cases, with up to 750 test cases. To balance representativeness and testing efficiency, this study introduces a combination testing theory model.

[0125] 43)Based on the combination testing theory and greedy algorithm, 50 groups of standardized test cases covering different fatigue actions are designed. Combination testing is an efficient testing method commonly used in software and system testing, which explores potential problems caused by the overlap of multiple factors by covering various combinations of input parameters. In this process, all target combinations are required to be covered at least once. The establishment of the combination testing model mainly needs to determine the influencing factors of the system under test and their values. The data used in this study are discrete data and there is no constraint between them. During testing, the test intensity k represents the coverage degree of the test. In this study, the test intensity k=2 is used to design two-factor combination cases. Under the condition of k=2, the greedy algorithm is used to cover all two-factor combinations, ensuring the recognition reliability and stability evaluation of the DMS system under the influence of multiple factors. The generation of test cases can use the greedy algorithm, which continuously preferentially enumerates test cases with high coverage, gradually obtaining the overall optimal solution. Each selection reduces the size of the problem. The selection logic of the greedy algorithm is as follows:

[0126] 1. Initialize data: set the coverage set to empty set, and all two-factor combinations are covered.

[0127] 2. In the candidate test cases, preferentially select the test cases that can cover the most two-factor combinations that have not appeared, and add them to the final test case set.

[0128] 3. Mark the two-factor combinations contained in the test case as covered.

[0129] 4. Repeat steps 2 and 3 until all two-factor combinations are covered at least once.

[0130] Finally, 50 groups of high-representative test cases are designed, which basically cover the common fatigue driving scenarios in actual driving, for subsequent testing.

[0131] 44) According to the above test cases, the test covers key fatigue actions such as closing eyes, abnormal head posture, yawning, etc. All tests are carried out in a consistent environment with controlled variables, real-time image acquisition using a camera, and simulation of relevant actions using a simulation robot. The detection rate, false alarm rate, average frame rate per second, and single frame processing time average of the DMS system are counted to evaluate the DMS system. The detection rate refers to the ratio of the number of times the system correctly detects abnormal behavior to the total number of times the abnormal behavior actually occurs, reflecting the ability of the DMS system to correctly identify abnormal behavior. The false alarm rate refers to the ratio of the number of times the system incorrectly identifies abnormal behavior to the total number of normal behaviors, indicating the frequency of the system incorrectly identifying normal behavior. The average frame rate per second reflects the processing speed and real-time performance. The single frame processing time reflects the computational efficiency of the DMS system.

[0132] The test is carried out in two ways. One is to use single-factor action test scenarios for testing, and the duration and angle of closing eyes, yawning, and abnormal head posture are set. Each single factor is tested at different durations and angles for a group of tests and repeated. The final system false alarm rate, detection rate, etc. are counted. The other is to use a multi-factor superimposed test scenario based on the greedy algorithm optimization for testing. Each test case is repeated for testing. The final system false alarm rate, detection rate, etc. are counted. In particular, for head posture abnormalities, the number of tests for lifting and lowering the head and turning the head left and right is the same and the average is taken. For multi-factor combination testing, the total detection rate and false detection rate are weighted averages of the detection rate and false detection rate of each action, and each action accounts for the same proportion. The test results provide data support and theoretical basis for identifying performance bottlenecks and optimization directions of the DMS system in fatigue and attention detection.

[0133] In summary, the driver monitoring system based on the simulation robot can effectively improve the test efficiency, the test accuracy is easier to control, and the random errors of real human actions are eliminated. The system can cover extreme scenarios and composite scenarios in the testing process, avoid safety hazards, has high controllability and repeatability, and helps to accurately evaluate system performance.

[0134] Although embodiments of the present application have been shown and described, it will be understood by those skilled in the art that various changes, modifications, substitutions and alterations can be made therein without departing from the principles and spirit of the application, the scope of which is defined by the appended claims and their equivalents.

Claims

1. A driver monitoring system test evaluation method based on a simulated robot, characterized by, The method comprises the following steps: Step 1: build a simulation robot for DMS test; Through 3D printing technology, the manufacturing of parts is completed, and the production of facial simulation skin is completed by using liquid silicone injection molding process; Step 2: design the control system of the DMS test simulation robot; Select different types of servo; adopt the I 2 C control scheme; through command analysis and task scheduling mechanism, realize fine control of multiple servo channels, drive the simulation robot to complete the standard face and head behavior Step 3: build a DMS system; A driver monitoring system based on visual perception is built, which uses a camera to collect video streams in real time, monitors the facial state of the simulation robot, extracts the parameters of eye closure degree, yawning frequency and head posture, and compares them with the set threshold to realize dynamic recognition and grading judgment of the driver's fatigue and distraction state; The DMS system comprises a face and eye key point detection module, an eye movement detection module, a head posture estimation module and an attention evaluation module; The face and eye key point detection module is used to extract and track the key point coordinates in the face region, and support the subsequent action recognition and posture estimation; The eye movement detection module is used to calculate the eye parameters based on the key points to judge the fatigue state; The head posture estimation module is used to estimate the three-dimensional posture angle of the head based on the key points to judge the distraction state; The attention evaluation module is used to comprehensively analyze the eye movement, head posture and other information, and output the attention dispersion state or warning of the driver; Step 4: generate and evaluate test cases; Based on the combination test theory and greedy algorithm, the standardized test cases covering different fatigue actions are designed, including single behavior and multi-behavior combination scenes, and the control instructions are issued through the serial port to drive the simulation robot to execute specific action sequences in real time. According to the comparison between the DMS system recognition result and the known action label, the performance index is calculated.

2. The method of claim 1, wherein the method is a method of evaluating a driver monitoring system test based on a simulated robot. The specific method of step 1 is as follows: 11) Structure design and 3D printing: design the head mechanical parts containing eyes, mouth, forehead and neck, and manufacture each part by using 3D printing technology; 12) Part assembly: assemble each part printed to form a complete head skeleton structure, and polish and lubricate the piston assembly; 13) Silicone face production and installation: use resin 3D printing mold, polish and polish, spray release agent, then inject liquid silicone, and cut the simulation face skin after demolding; The silicone face is firmly installed on the robot face structure by adhesion.

3. The method of claim 1, wherein the method further comprises: The specific method of step 2 is as follows: 21) According to the required torsion angle and torsion torque of different structural parts, select and arrange different rudders; 22)System supports up to 16 rudder tasks to be executed simultaneously, each task contains rudder channel number, starting angle, target angle, holding time, forward time and return time; the adjustment of rudder speed is based on time interpolation algorithm, each rudder action is divided into several small steps, and the target angle is approached gradually in each time slice, simulating natural human action; the system supports to keep fixed time after completing the action, and supports to automatically return to the initial angle, realizing the action closed loop; the user inputs specific format instructions through the serial port, and the system automatically parses and generates the task list; the angle is mapped to standard PWM pulse width by using the angleToMicroseconds() function, and 4096 resolution high precision PWM signal is output through the PCA9685 module, realizing the precise control of rudder angle.

4. The method of claim 3, wherein the method further comprises: The specific method of step 21) is as follows: JX PDI-1109MG: for small torque; JX PDI-6221MG: for medium torque; HS-805BB: for heavy load.

5. The method of claim 1, wherein the method further comprises: The specific method of step three is as follows: 31) Image acquisition and preprocessing, OpenCV is called to obtain video stream through camera, and grayscale conversion, channel reconstruction and bilateral filter denoising are performed; 32) Integrate MediaPipe Face Mesh model to detect 468 facial key points in real time; first, the model inference outputs normalized 3D coordinates; the 3D coordinate sequence is represented in the normalized space defined by formula (1): In the formula, w and h are the width and height of the input image; d ref The reference depth value is based on the face proportion; through the linear mapping of formula (2), the normalized coordinates are converted into the absolute position in the image pixel coordinate system: (x norm y norm , z rel x, y represent the coordinates of the original facial key points in the 3D coordinate system; x, y represent the pixel positions of the key points on the image plane. z is the depth value of the key point relative to a certain reference plane, usually the relative depth obtained by model inference, used to describe the stereo geometric features of the face structure; where (x pixel , y pixel ) are the image pixel coordinates of the normalized coordinates after being restored to the original image size by linear mapping, used for accurate positioning and visualization of the key points in the image. Enumerate and filter all facial key points, including eye key points, iris key points and pose estimation key points, after filtering the key points, the system first visualizes the key points related to the eyes in the image to assist in debugging and result verification; at the same time, the head pose estimation module estimates the head pose based on the selected facial key points, and draws three-dimensional coordinate axes in the image to intuitively show the orientation and angle change of the head; 33) After completing the extraction of facial key points and coordinate optimization, the system calculates the multi-modal features of eye movement, gaze direction and head pose based on the key point sequence; according to the selected 6 key points of the left and right eyes, the aspect ratio of the eyes is calculated, and the aspect ratio ratio is calculated by formula (3) to obtain the Euclidean distance relationship between the key points in the vertical and horizontal directions of the eyes: In the formula, P1, P2…P6 are the coordinates of the left and right eye key points, which are used to calculate the aspect ratio of the eyes; After obtaining the EAR value of the left and right eyes, the average value of the EAR values of the left and right eyes is calculated, if the average value is less than 0.2, it is marked as closed eye state, the proportion of closed eye time in the unit time window PERCLOS is calculated, the fatigue degree of the driver is judged through the proportion PERCLOS, if the proportion PERCLOS exceeds the set threshold, it is determined that the driver is in a fatigue state, and the system will trigger a warning. The iris center points of the left and right eyes are located, and a local eye coordinate system is established in combination with the corner key points; the relative position of the iris in the white eye area is estimated by comparing the offset of the iris center relative to the corner, and then the line-of-sight direction vector is derived; the line-of-sight direction vector describes the user's current gaze direction as being straight ahead, left, right, up, or down; The system estimates the three-dimensional attitude angle of the head through a key point-based geometric modeling method; 34) The driver monitoring function is decomposed into multiple independent functional modules, including a face and eye key point detection module, an eye movement detection module, a head pose estimation module, and an attention evaluation module; The face and eye key point detection module is used to extract and track the key point coordinates in the face region, supporting subsequent action recognition and pose estimation; The eye movement detection module is used to calculate the eye aspect ratio, identify the closed eye state, and determine the line-of-sight direction; The head pose estimation module is used to estimate the Euler angle of the head and determine the orientation change; The attention evaluation module is used to comprehensively analyze the eye movement and head pose information and output the driver's attention distraction state or warning.

6. A test evaluation method for a driver monitoring system based on a simulated robot according to claim 5, characterized in that The specific method for the system to estimate the three-dimensional attitude angle of the head through a key point-based geometric modeling method in step 33) is as follows: The system selects face key points that are not affected by expression changes and have consistent structure semantics from the 468 face key points extracted by MediaPipe FaceMesh as reference points; after the key points are extracted, the head pose estimation module estimates the three-dimensional attitude angle of the head based on the PnP method; specifically, first, the known face three-dimensional model points are matched with the two-dimensional projection coordinates in the image, and the internal parameters of the camera are used to call the cv2.solvePnP() function provided by OpenCV to solve the rotation vector R and the translation vector t; the three-dimensional-two-dimensional point matching is realized by minimizing the re-projection error problem of formula (4), and then the attitude information of the head in the three-dimensional space is calculated; where π(·) is the camera projection function; K is the intrinsic matrix (contains focal length f, principal point (cx, cy)); X i is the three-dimensional face model key point coordinate; x i is the corresponding two-dimensional pixel coordinate; the rotation vector is then converted into a rotation matrix via cv2.Rodrigues(), which is further decomposed to obtain three Euler angle parameters of the head, and the Euler angles are solved by formula (5): where r ij is the element in the ith row and jth column of the rotation matrix R; arctan2(y,x) is the four quadrant arctangent function, output range is (-π, π], converted from radians to degrees by arctan2(y,x) = atan2(y,x) * 180 / π The system sets an offset threshold for each angle to determine whether the driver has non-concentrated behavior; when any attitude angle continuously exceeds the set range and exceeds the time threshold, the AttentionScorer module outputs the attention distraction state, triggering the corresponding prompt or warning mechanism.

7. The method of claim 1, wherein the method further comprises: The specific method of step four is as follows: 41) A test action combination containing multi-dimensional actions such as eye opening and closing, mouth opening and closing, head twisting, and head tilting is designed to build a systematic test case set; the test scheme sets a duration gradient of 0.5s and 1s with a difference of 0.5s; The multi-time dimension test scheme is based on the complexity of actual driving scenarios: short-term eye closure, i.e., <0.5s, is within the normal blinking range, while continuous eye closure, i.e., ≥2s, is considered a potential sign of fatigue driving; looking down for 1 second is a short-term gaze at the instrument panel or vehicle display screen, and if the duration exceeds 3 seconds, there is a risk of driving; For the eye closure time, 2s is selected as the center point, and the difference time interval of 0.5s and 1s is selected. Five test parameter values of 0.5, 1, 2, 2.5, and 3 are selected. For the yawp duration, five parameters of 0.5, 1, 3, 3.5, and 4 are selected. The parameter values during the head posture abnormality test are set according to the definition of head posture abnormality in the national standard; 42) Multi-element combination test is introduced, focusing on the interaction of three core behavior dimensions: ① eye closure, ② mouth opening, ③ head posture deviation; If the three behaviors appear simultaneously or sequentially, it indicates that the driving state is abnormal; 43) The data used in the combined test theoretical model are all discrete data and there is no constraint between them. In the test process, the test intensity k represents the coverage degree of the test. The test intensity k=2 is used to design the two-factor combination use case. Under the condition of k=2, the greedy algorithm is used to cover all two-parameter combinations; 44) According to the test cases, including eye closure, head posture abnormality, and yawp, the test is carried out; All tests are carried out in an environment with consistent control variables. Real-time image acquisition is performed using a camera, and a simulation robot is used to simulate relevant actions. The detection rate, false alarm rate, average frame rate per second, and average single-frame processing time of the DMS system are calculated to evaluate the DMS system.

8. The method of claim 7, wherein the method further comprises: The logic of the greedy algorithm in step 43) is as follows: 431) Initialize data: set the coverage set to an empty set, and all two-parameter combinations are covered; 432) In the candidate test cases, select the one that can cover the most two-parameter combinations that have not appeared, and add it to the final test case set; 433) Mark the two-parameter combinations contained in the use case as covered; 434) Repeat steps 422) and 423) until all two-parameter combinations are covered at least once.

Images