A method of monitoring the head posture of a vehicle driver

Abstract

The application relates to a vehicle driver head posture monitoring method, which comprises the following steps: obtaining a driver face image through an infrared camera; extracting six 2D face key points based on the driver face image and marking 2D image coordinate points; building a 3D standard face model, extracting 3D standard face key points and marking 3D model coordinate points; solving a rotation matrix based on the 2D image coordinate points, the 3D model coordinate points, the focal length, the focal point and the distortion coefficient of the infrared camera; solving driver head posture Euler angles according to the rotation matrix; performing face three-dimensional angle regression according to the head posture Euler angles, judging whether the driver head posture changes or not; judging whether the driver exceeds a safe driving range according to the driver head posture change, if not, continuously monitoring; if yes, warning the driver; the application can monitor the driver head posture in real time, reduce the accident rate and ensure driving safety.

Description

Technical Field

[0001] This invention relates to the field of automotive technology, and in particular to a method for monitoring the head posture of a vehicle driver. Background Technology

[0002] Prolonged driving can harm a driver's physical and mental health, easily leading to various traffic accidents and affecting driving safety. While traffic accidents are caused by many factors, the main cause is improper driving behavior, such as fatigued driving and distracted driving. Therefore, unsafe driving behavior is a problem that cannot be ignored and requires real-time monitoring of driver safety. Summary of the Invention

[0003] The purpose of this invention is to provide a method for monitoring the head posture of a vehicle driver, addressing the shortcomings of existing technologies.

[0004] This invention is achieved using the following technical solution:

[0005] A method for monitoring the head posture of a vehicle driver includes the following steps:

[0006] Install an infrared camera to capture the driver's facial image;

[0007] Based on the driver's face image, six 2D facial key points were extracted, and the coordinates of the 2D image points were marked.

[0008] A 3D standard face model is to be constructed, key points of the 3D standard face are extracted, and the coordinate points of the 3D model are marked.

[0009] The rotation matrix is ​​calculated based on the coordinate points of the 2D image, the coordinate points of the 3D model, the focal length, focus, and distortion coefficients of the infrared camera; the Euler angles of the driver's head posture are then calculated based on the rotation matrix.

[0010] Based on the Euler angles of head posture, a 3D angle regression of the face is performed to determine whether there is a change in the driver's head posture.

[0011] Based on changes in the driver's head posture, determine whether the driver has exceeded the safe driving range. If not, continue monitoring; if so, issue a warning to the driver.

[0012] As a further explanation of the invention, an infrared camera is set up to acquire images of the driver's behavioral characteristics; including:

[0013] Install infrared cameras to acquire images of driver behavior characteristics;

[0014] Based on the driver's behavioral feature image, occlusion detection is performed first, then face detection is performed to identify the facial feature image, and the face frame is cropped to retain the image containing the face, thereby obtaining the driver's face image.

[0015] As a further illustration of the invention, based on the driver's facial image, 2D facial key points are extracted and 2D image coordinate points are marked; including:

[0016] The 2D facial key points include eyebrows, eyes, nose, mouth, and facial contours; and the coordinates of the left and right corners of the eyes, nose, left corner of the mouth, left corner of the mouth, and chin are marked with 2D image coordinates.

[0017] As a further explanation of the invention, a 3D standard face model is proposed, key points of the 3D standard face are extracted, and the coordinate points of the 3D model are marked; including:

[0018] The key points of a 3D standard human face include eyebrows, eyes, nose, mouth, and facial contours; and the coordinates of the 3D model are marked for the left corner of the eye, the right corner of the eye, the nose, the left corner of the mouth, the right corner of the mouth, and the chin.

[0019] As a further explanation of the invention, a rotation matrix is ​​calculated based on 2D image coordinate points, 3D model coordinate points, and the focal length, focus, and distortion coefficients of the infrared camera; the Euler angles of the driver's head posture are then solved based on the rotation matrix; including:

[0020] The rotation and translation vectors are calculated based on the coordinate points of the 2D image, the coordinate points of the 3D model, the focal length, focus, and distortion coefficient of the infrared camera; the rotation matrix is ​​calculated based on the rotation and translation vectors; and the Euler angles of the driver's head posture are obtained by transforming the rotation matrix.

[0021] As a further explanation of the invention, facial 3D angle regression is performed based on Euler angles of head posture to determine whether there is a change in the driver's head posture; including:

[0022] Map the coordinates of the 3D model of the nose position onto a 2D plane to obtain the 2D mapped coordinates. Connect the 2D image coordinates of the nose position with the 2D mapped coordinates. Determine whether there is a change in the driver's head posture and the degree of change based on the length of the connection.

[0023] As a further illustration of the invention, the infrared camera is positioned inside the cab directly facing the driver.

[0024] As a further illustration of the invention, the infrared camera is installed at the A-pillar inside the driver's cab.

[0025] Compared with the prior art, the present invention has the following beneficial technical effects:

[0026] This invention monitors the driver's head posture in real time to determine if the driver is engaging in unsafe driving behavior, and provides timely warnings when the driver exhibits such behavior, thereby reducing the accident rate and ensuring driving safety. Attached Figure Description

[0027] Figure 1 This is a flowchart illustrating the present invention. Detailed Implementation

[0028] Example 1

[0029] like Figure 1 As shown, a method for monitoring the head posture of a vehicle driver includes the following steps:

[0030] Install an infrared camera to capture the driver's facial image;

[0031] Based on the driver's face image, six 2D facial key points were extracted, and the coordinates of the 2D image points were marked.

[0032] A 3D standard face model is to be constructed, key points of the 3D standard face are extracted, and the coordinate points of the 3D model are marked.

[0033] The rotation matrix is ​​calculated based on the coordinate points of the 2D image, the coordinate points of the 3D model, the focal length, focus, and distortion coefficients of the infrared camera; the Euler angles of the driver's head posture are then calculated based on the rotation matrix.

[0034] Based on the Euler angles of head posture, a 3D angle regression of the face is performed to determine whether there is a change in the driver's head posture.

[0035] Based on changes in the driver's head posture, determine whether the driver has exceeded the safe driving range. If not, continue monitoring; if so, issue a warning to the driver.

[0036] Furthermore, an infrared camera is installed to acquire images of the driver's behavioral characteristics; including:

[0037] Install infrared cameras to acquire images of driver behavior characteristics;

[0038] Based on the driver's behavioral characteristic image, occlusion detection is first performed, followed by face detection to identify facial features and crop the facial outline, retaining only the image containing the face, thus obtaining the driver's facial image. In other words, only images containing the face are retained.

[0039] Furthermore, based on the driver's facial image, 2D facial key points are extracted and the 2D image coordinates are marked; including:

[0040] 2D facial landmarks include eyebrows, eyes, nose, mouth, and facial contours; and 2D image coordinates are marked for the left and right corners of the eyes, nose, left and right corners of the mouth, and chin. Preferably, the PFLD_Ultralight_112_1 (facial landmark) algorithm model is used for facial landmark detection, resulting in 98 facial landmarks. From these 98 landmarks, 6 landmarks—left and right corners of the eyes, nose, left and right corners of the mouth, and chin—are extracted and marked.

[0041] Furthermore, a 3D standard face model will be constructed, key points of the 3D standard face will be extracted, and the coordinates of the 3D model will be marked; including:

[0042] The key points of a 3D standard human face include eyebrows, eyes, nose, mouth, and facial contours; and the coordinates of the 3D model are marked for the left corner of the eye, the right corner of the eye, the nose, the left corner of the mouth, the right corner of the mouth, and the chin.

[0043] Furthermore, the rotation matrix is ​​calculated based on the coordinate points of the 2D image, the coordinate points of the 3D model, and the focal length, focus, and distortion coefficients of the infrared camera; the Euler angles of the driver's head pose are then solved based on the rotation matrix; including:

[0044] The rotation and translation vectors are calculated based on the coordinates of the 2D image, the coordinates of the 3D model, the focal length, focus, and distortion coefficients of the infrared camera. A rotation matrix is ​​then calculated using these vectors. Finally, the Euler angles of the driver's head pose are obtained from the rotation matrix. Since the infrared camera captures the driver's face image with a bounding box, the focal length is approximated by the image size, and the focus is approximated by the image center point. Preferably, the `solvePnP` function from the OpenCV library (a computer vision library) is used to calculate the rotation and translation vectors, and the rotation matrix is ​​calculated based on these vectors. The Euler angles of the driver's head pose are then obtained from the rotation matrix.

[0045] Furthermore, based on the Euler angles of the head posture, a 3D facial angle regression is performed to determine whether there are any changes in the driver's head posture; including:

[0046] The coordinates of the 3D model of the nose are mapped onto a 2D plane to obtain 2D mapped coordinates. A line is then drawn connecting the 2D image coordinates of the nose to the 2D mapped coordinates. The length of this line is used to determine whether the driver's head posture changes and the degree of change. This, in turn, helps determine if the driver is engaging in unsafe driving behavior.

[0047] Furthermore, an infrared camera is installed inside the cab, directly facing the driver's position.

[0048] Example 2

[0049] Implementation 2 is largely the same as Implementation 1, with the following differences:

[0050] The infrared camera is installed on the A-pillar inside the driver's cab.

[0051] It will be apparent to those skilled in the art that the present invention is not limited to the details of the exemplary embodiments described above, and that the invention can be implemented in other specific forms without departing from its spirit or essential characteristics. Therefore, the embodiments should be considered in all respects as exemplary and non-limiting, and the scope of the invention is defined by the appended claims rather than the foregoing description. Thus, all variations falling within the meaning and scope of equivalents of the claims are intended to be included within the present invention. No reference numerals in the claims should be construed as limiting the scope of the claims.

[0052] The above-described embodiments are only used to illustrate the technical solutions of the present invention, and are not intended to limit it. Although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features. Such modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. A method for monitoring the head posture of a vehicle driver, characterized in that, Includes the following steps: The driver's facial image is obtained through an infrared camera; Based on the driver's face image, six 2D facial key points were extracted, and the coordinates of the 2D image points were marked. A 3D standard face model is to be constructed, key points of the 3D standard face are extracted, and the coordinate points of the 3D model are marked. The rotation matrix is ​​calculated based on the coordinate points of the 2D image, the coordinate points of the 3D model, and the focal length, focus, and distortion coefficients of the infrared camera. Solve for the Euler angles of the driver's head posture using the rotation matrix; Based on the Euler angles of head posture, a 3D angle regression of the face is performed to determine whether there is a change in the driver's head posture. Based on changes in the driver's head posture, determine whether the driver has exceeded the safe driving range; if not, continue monitoring. If so, a warning will be issued to the driver.

2. The vehicle driver head posture monitoring method as described in claim 1, characterized in that, Set up an infrared camera to acquire images of driver behavior characteristics; including: Install infrared cameras to acquire images of driver behavior characteristics; Based on the driver's behavioral feature image, occlusion detection is performed first, then face detection is performed to identify the facial feature image, and the face frame is cropped to retain the image containing the face, thereby obtaining the driver's face image.

3. The vehicle driver head posture monitoring method as described in claim 2, characterized in that, Based on the driver's facial image, six 2D facial landmarks were extracted, and the coordinates of the 2D image points were marked; including: The 2D facial key points include eyebrows, eyes, nose, mouth, and facial contours; and the coordinates of the left and right corners of the eyes, nose, left corner of the mouth, left corner of the mouth, and chin are marked with 2D image coordinates.

4. The vehicle driver head posture monitoring method as described in claim 1, characterized in that, A 3D standard face model will be constructed, key points of the 3D standard face will be extracted, and the coordinates of the 3D model will be marked; including: The key points of a 3D standard human face include eyebrows, eyes, nose, mouth, and facial contours; and the coordinates of the 3D model are marked for the left corner of the eye, the right corner of the eye, the nose, the left corner of the mouth, the right corner of the mouth, and the chin.

5. The vehicle driver head posture monitoring method as described in claims 3 and 4, characterized in that, The rotation matrix is ​​calculated based on the coordinate points of the 2D image, the coordinate points of the 3D model, and the focal length, focus, and distortion coefficients of the infrared camera. Solving the Euler angles of the driver's head posture based on the rotation matrix; including: The rotation and translation vectors are calculated based on the coordinate points of the 2D image, the coordinate points of the 3D model, the focal length, focus, and distortion coefficients of the infrared camera; the rotation matrix is ​​calculated based on the rotation and translation vectors; and the Euler angles of the driver's head posture are obtained by transforming the rotation matrix.

6. The vehicle driver head posture monitoring method as described in claim 5, characterized in that, Based on Euler angles of head posture, 3D facial angle regression is performed to determine whether there are any changes in the driver's head posture; including: Map the 3D model coordinates of the nose position onto a 2D plane to obtain 2D mapped coordinates. Connect the 2D image coordinates of the nose position with the 2D mapped coordinates. Determine whether there is a change in the driver's head posture and the degree of change based on the length of the connection.

7. The vehicle driver head posture monitoring method as described in claim 1, characterized in that, The infrared camera is positioned inside the driver's cab, directly facing the driver.

8. The vehicle driver head posture monitoring method as described in claim 1, characterized in that, The infrared camera is installed on the A-pillar inside the driver's cab.

Images