Driver multi-part detection method and device, electronic control unit and vehicle

Abstract

The application is suitable for the field of intelligent traffic safety technology, and provides a driver multi-part detection method, device, electronic control unit and vehicle. The method comprises: acquiring a driver image; performing target part detection on the driver image to determine a target region, wherein the target part detection comprises face detection and hand detection, and the target region comprises any one or more of a face region and a hand region; determining an upper body detection region according to the target region, and determining an upper body posture of the driver based on the driver upper body detected through the upper body detection region; determining an optimal target region according to the upper body posture and the target region, and detecting target parts of the driver based on the optimal target region, wherein the target parts comprise any one or more of a face and a hand. The application can avoid the problems of face or hand mis-detection or inaccurate detection, and improve the detection accuracy of multi-part detection.

Description

Technical Field

[0001] This application relates to the field of intelligent transportation safety technology, specifically to a method, device, electronic control unit, and vehicle for detecting multiple parts of a driver. Background Technology

[0002] With the continuous development of automotive technology, car cabins are becoming increasingly intelligent. Driver Monitoring Systems (DMS) can analyze driver attributes and behaviors to monitor driver fatigue, distraction, and dangerous behaviors.

[0003] When monitoring driver fatigue and distraction, the DMS needs to perform facial detection to determine whether the driver is fatigued or distracted. When monitoring dangerous behavior, it needs to perform hand detection to determine whether the driver is engaging in dangerous behavior. The aforementioned facial detection and hand detection are collectively referred to as multi-site detection.

[0004] However, due to the instability of the DMS detector, interference can easily occur between face detection and hand detection. That is, the face detector may detect a hand as a face, or the hand detector may detect a face as a hand, resulting in false detection or inaccurate detection of face or hand. Summary of the Invention

[0005] This application provides a method, device, electronic control unit, and vehicle for detecting multiple parts of a driver's body, which can avoid the problems of false detection or inaccurate detection of faces or hands, and improve the detection accuracy of multiple parts detection.

[0006] This application is achieved through the following technical solution:

[0007] In a first aspect, embodiments of this application provide a method for detecting multiple body parts of a driver, including:

[0008] Acquire driver image;

[0009] Target part detection is performed on the driver image to determine the target region. Target part detection includes face detection and hand detection. The target region includes any one or more of the face region and the hand region.

[0010] The upper body detection area is determined based on the target area, and the driver's upper body posture is determined based on the driver's upper body detected through the upper body detection area.

[0011] Based on the upper body posture and the target area, the optimal target area is determined, and the target parts of the driver are detected based on the optimal target area. The target parts include any one or more of the face and hands.

[0012] In one possible implementation of the first aspect, determining the upper body detection area based on the target area includes:

[0013] If a face region exists, the upper body detection region is determined based on the face region and the preset head-to-body ratio coefficient; if no face region exists, the driver image is used as the upper body detection region.

[0014] In one possible implementation of the first aspect, the DBFace network model is used to detect target parts in the driver image to determine the target region, and to detect the head orientation in the driver image to determine the head orientation.

[0015] The upper body detection area is determined based on the face region and a preset head-to-body ratio coefficient, including:

[0016] The face region is rotated according to the direction the head is facing, so that the head in the face region is facing positive.

[0017] The upper body detection area is determined based on the rotated face area and the preset head-to-body ratio coefficient.

[0018] In one possible implementation of the first aspect, the DBFace network model performs head orientation detection on a driver image to determine the head orientation, including:

[0019] Based on the multi-head attention mechanism, a head orientation detection branch is added after the feature extraction layer of the DBFace network model to detect and determine the head orientation; the head orientation includes forward, backward, leftward, and rightward.

[0020] In one possible implementation of the first aspect, determining the upper body detection area based on the rotated face region and a preset head-to-body ratio coefficient includes:

[0021] The width and height of the upper body detection area are determined based on the width and height of the rotated face area and the preset head-to-body ratio coefficient.

[0022] The width and height of the upper body detection area are:

[0023] Width half-body =8 × Width face +0.1×Width face

[0024] Height half-body =4×Height face +0.1×Height face

[0025] Among them, Width half-body Height is the width of the upper body detection area.half-body Width is the height of the upper body detection area. face Height is the width of the rotated face region. face This represents the height of the rotated face region.

[0026] In one possible implementation of the first aspect, determining the driver's upper body posture based on the driver's upper body detected by the upper body detection area includes:

[0027] A Yolox-S network model was used to detect the driver's upper body in the upper body detection area.

[0028] When the driver's upper body is detected, the OpenPose network model is used to estimate the pose of the driver's upper body and determine the upper body skeleton points. The upper body skeleton points include the center point of the target part, which includes any one or more of the center points of the face and the hand.

[0029] In one possible implementation of the first aspect, an optimal target region is determined based on the upper body posture and the target region, and the target parts of the driver are detected based on the optimal target region, including:

[0030] The center point of the face region is determined based on the rotated face region, and the center point of the hand region is determined based on the hand region.

[0031] The optimal face center point is determined based on the face center point and the face region center point, and the optimal face region is determined based on the optimal face center point and the face region.

[0032] And / or, determine the optimal hand center point based on the hand center point and the hand region center point, and determine the optimal hand region based on the optimal hand center point and the hand region;

[0033] The driver's face is detected based on the optimal face region, and / or the driver's hands are detected based on the optimal hand region.

[0034] In one possible implementation of the first aspect, the driver multi-site detection method further includes:

[0035] If the face and hand areas are not detected, or if the upper body is not detected, the driver image is reacquired and detected again.

[0036] Secondly, embodiments of this application provide a method for detecting abnormal driving behavior of a driver, including:

[0037] According to the driver multi-part detection method described in any one of the first aspects, multiple sets of optimal target regions are determined, and the optimal target regions include any one or more of the optimal face region and the optimal hand region;

[0038] Using the first deep learning model, it determines whether the changing trend of facial contour points in multiple optimal facial regions matches the fitted sample and whether the duration reaches the preset time. If so, it outputs a fatigue driving signal and an alarm signal. It calculates the facial pose angle based on the changing trend of facial contour points in multiple optimal facial regions and determines whether the facial pose angle exceeds the preset angle threshold. If so, it outputs a distracted driving signal and an alarm signal.

[0039] And / or, a second deep learning model is used to determine whether a target object exists in multiple optimal hand regions, and if a target object exists, the category of the target object is determined; based on the category of the target object, the dangerous behavior category and alarm signal are output.

[0040] Thirdly, embodiments of this application provide a driver multi-site detection device, including:

[0041] The acquisition module is used to acquire images of the driver.

[0042] The detection module is used to detect target parts in the driver image and determine the target region. The target part detection includes face detection and hand detection, and the target region includes any one or more of the face region and the hand region.

[0043] The first determining module is used to determine the upper body detection area based on the target area, and to determine the driver's upper body posture based on the driver's upper body detected through the upper body detection area.

[0044] The output module is used to determine the optimal target area based on the upper body posture and the target area, and to detect the driver's target parts based on the optimal target area. The target parts include any one or more of the face and hands.

[0045] Fourthly, embodiments of this application provide a driver abnormal driving behavior detection device, comprising:

[0046] The second determining module is used to determine multiple sets of optimal target regions according to the driver multi-part detection method described in any one of the first aspects, wherein the optimal target regions include any one or more of the optimal face region and the optimal hand region;

[0047] The execution module is used to employ a first deep learning model to determine whether the changing trends of facial contour points included in multiple optimal facial regions conform to the fitted samples and whether the duration reaches a preset time, and outputs a fatigue driving signal and an alarm signal when they conform to the fitted samples and the duration reaches the preset time; calculates the facial pose angle based on the changing trends of facial contour points included in multiple optimal facial regions, and determines whether the facial pose angle exceeds a preset angle threshold, and outputs a distracted driving signal and an alarm signal when it exceeds the preset angle threshold; and / or, employs a second deep learning model to determine whether there is a target object in multiple optimal hand regions, and determines the category of the target object when it exists; outputs a dangerous behavior category and an alarm signal based on the category of the target object.

[0048] Fifthly, embodiments of this application provide an electronic control unit, including a memory and a processor. The memory stores a computer program that can run on the processor. When the processor executes the computer program, it implements the steps of the driver multi-part detection method as described in any of the first aspects, and / or the steps of the driver abnormal driving behavior detection method as described in the second aspect.

[0049] Sixthly, embodiments of this application provide a vehicle including the electronic control unit as described in the fifth aspect.

[0050] In a seventh aspect, embodiments of this application provide a computer-readable storage medium storing a computer program that, when executed by a processor, implements the steps of the driver multi-part detection method as described in any of the first aspects, and / or the steps of the driver abnormal driving behavior detection method as described in the second aspect.

[0051] Eighthly, embodiments of this application provide a computer program product that, when running on an electronic control unit, causes the electronic control unit to perform the steps of the driver multi-part detection method described in any of the first aspects, and / or the steps of the driver abnormal driving behavior detection method described in the second aspect.

[0052] It is understood that the beneficial effects of aspects two through eight above can be found in the relevant descriptions in aspect one above, and will not be repeated here.

[0053] The driver multi-part detection method, device, electronic control unit, and vehicle provided in this application acquire a driver image, perform target part detection on the driver image, and determine the target region. The target part detection includes face detection and hand detection. The target region includes any one or more of the face region and hand region. An upper body detection region is determined based on the target region. The driver's upper body posture is determined based on the upper body detected by the upper body detection region. An optimal target region is determined based on the upper body posture and the target region. The driver's target parts are detected based on the optimal target region. The target parts include any one or more of the face and hands. This can avoid the problem of false detection or inaccurate detection of face or hand and improve the detection accuracy of multi-part detection.

[0054] It should be understood that the above general description and the following detailed description are exemplary and explanatory only, and are not intended to limit this specification. Attached Figure Description

[0055] To more clearly illustrate the technical solutions in the embodiments of this application, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0056] Figure 1 This is a diagram illustrating false positives for faces and hands.

[0057] Figure 2 This is a schematic diagram illustrating an application scenario of the multi-part detection method for drivers provided in an embodiment of this application;

[0058] Figure 3 This is a schematic flowchart of a multi-part detection method for drivers provided in an embodiment of this application;

[0059] Figure 4 This is a schematic flowchart of a multi-part detection method for drivers provided in an embodiment of this application;

[0060] Figure 5 This is a schematic diagram of the orientation of a person's head according to an embodiment of this application;

[0061] Figure 6 This is a schematic flowchart of a multi-part detection method for drivers provided in an embodiment of this application;

[0062] Figure 7 This is a schematic flowchart of a multi-part detection method for drivers provided in an embodiment of this application;

[0063] Figure 8This is a schematic diagram of the upper body skeleton points provided in one embodiment of this application;

[0064] Figure 9 This is a schematic flowchart of a multi-part detection method for drivers provided in an embodiment of this application;

[0065] Figure 10 This is a schematic flowchart of a method for detecting abnormal driving behavior of a driver provided in an embodiment of this application;

[0066] Figure 11 This is a schematic diagram of the structure of a driver multi-site detection device provided in one embodiment of this application;

[0067] Figure 12 This is a schematic diagram of the structure of a driver abnormal driving behavior detection device provided in an embodiment of this application;

[0068] Figure 13 This is a schematic diagram of the structure of an electronic control unit provided in an embodiment of this application;

[0069] Figure 14 This is a schematic diagram of the structure of a vehicle provided in one embodiment of this application. Detailed Implementation

[0070] The present application will be described more clearly below with reference to specific embodiments. These embodiments will help those skilled in the art to further understand the function of the present application, but do not limit the present application in any way. It should be noted that those skilled in the art can make several modifications and improvements without departing from the concept of the present application. These all fall within the protection scope of the present application.

[0071] It should be understood that, when used in this application specification and the appended claims, the term "comprising" indicates the presence of the described features, integrals, steps, operations, elements and / or components, but does not exclude the presence or addition of one or more other features, integrals, steps, operations, elements, components and / or a collection thereof.

[0072] It should also be understood that the term “and / or” as used in this application specification and the appended claims means any combination of one or more of the associated listed items and all possible combinations, and includes such combinations.

[0073] In the description of this application and the appended claims, the terms "first," "second," "third," etc., are used only to distinguish descriptions and should not be construed as indicating or implying relative importance.

[0074] References to "one embodiment" or "some embodiments" as described in this specification mean that one or more embodiments of this application include a specific feature, structure, or characteristic described in connection with that embodiment. Therefore, the phrases "in one embodiment," "in some embodiments," "in other embodiments," "in still other embodiments," etc., appearing in different parts of this specification do not necessarily refer to the same embodiment, but rather mean "one or more, but not all, embodiments," unless otherwise specifically emphasized. The terms "comprising," "including," "having," and variations thereof mean "including but not limited to," unless otherwise specifically emphasized.

[0075] Furthermore, the term "multiple" mentioned in the embodiments of this application should be interpreted as two or more.

[0076] With the continuous development of automotive technology, car cabins are becoming increasingly intelligent. As a crucial player in the vehicle, the driver's attributes and behaviors are of paramount importance. A Driver Monitor System (DMS) analyzes driver attributes and behaviors to monitor for driver fatigue, distraction, or dangerous behaviors. Dangerous behaviors include actions that impair driver attention, such as smoking, using a mobile phone, or eating. In short, DMS can monitor driver fatigue, distraction, and dangerous driving behaviors, thereby preventing traffic accidents caused by poor driving habits.

[0077] Specifically, when monitoring fatigue, the DMS (Driver Monitoring System) samples the driver's eye-closing and yawning behaviors using cameras, and combines this data with factors such as driving time and speed to determine driver fatigue and its level. When monitoring distraction, the DMS samples the driver's gaze deviation and facial angle deviation using cameras, and determines whether the driver's gaze or facial angle has deviated based on a deviation angle threshold. A timer is started from the trigger threshold, and the distraction level is determined based on the duration of the deviation. When monitoring dangerous driving behavior, the DMS samples the driver's hand gestures using cameras to determine if the driver is smoking, making phone calls, or eating. When fatigued driving, distracted driving, or dangerous driving behavior is detected, appropriate alerts are provided, such as buzzer alarms, voice alarms, seatbelt tightening, and instrument panel warnings.

[0078] As mentioned above, when monitoring driver fatigue and distraction, the DMS (Driver Monitoring System) needs to perform face detection to determine if the driver is fatigued or distracted. When monitoring dangerous driving behavior, it needs to perform hand detection to determine if the driver is engaging in dangerous driving behavior. These face and hand detections are collectively referred to as multi-site detection. However, due to the instability of the DMS detector, interference can easily occur between face and hand detection. Specifically, when a hand is close to the camera, or a face is close to the camera, or a face and hand are close together, the face detector may misdetect a hand as a face, or the hand detector may misdetect a face as a hand (see [link to relevant documentation]). Figure 1 This phenomenon leads to false detections or inaccurate detection of faces or hands.

[0079] Based on the above problems, the inventors discovered through research that by detecting the driver's upper body posture, the driver's upper body skeletal points can be determined, and then the face detection and hand detection can be optimized based on these upper body skeletal points, that is, the detection of multiple parts can be optimized.

[0080] In other words, this application embodiment acquires a driver image, performs target part detection on the driver image, and determines the target region. The target part detection includes face detection and hand detection. The target region includes any one or more of the face region and hand region. An upper body detection region is determined based on the target region. The driver's upper body posture is determined based on the upper body detected by the upper body detection region. Based on the upper body posture and the target region, an optimal target region is determined. The driver's target parts are detected based on the optimal target region. The target parts include any one or more of the face and hand. This can avoid the problem of false detection or inaccurate detection of face or hand and improve the detection accuracy of multi-part detection.

[0081] Figure 2 This is a schematic diagram illustrating an application scenario of a multi-part detection method for drivers provided in an embodiment of this application. For example... Figure 2 As shown, in a real-world scenario, the electronic control unit 10 acquires an image of the driver 30 captured by the camera 20, and performs target part detection on the driver image. Based on the obtained target area, an upper body detection area is determined. The target part detection includes face detection and hand detection. The target area includes any one or more of the face area and hand area. Based on the driver's upper body detected through the upper body detection area, the driver's upper body posture is determined. Based on the upper body posture, the optimal target area is finally determined, and the driver's target parts are detected based on the optimal target area.

[0082] Figure 3 This is a schematic flowchart of a multi-site detection method for drivers provided in an embodiment of this application. Figure 3As shown, the method in the embodiments of this application may include:

[0083] Step 101: Obtain the driver's image.

[0084] Optionally, during driving, the driver's image can be acquired via a DMS camera, where the DMS camera can be a near-infrared camera, and the driver's image acquired by the DMS camera can be in YUV data format.

[0085] Step 102: Detect target parts in the driver image to determine the target area.

[0086] Optionally, target region detection includes face detection and hand detection, and the target region includes any one or more of the face region and hand region. A deep learning model is used to perform face and hand detection on the acquired driver image. For example, in this embodiment, an anchor-free DBFace network model is used to perform face and hand detection on the driver image to determine any one or more of the face region and hand region. Before using the DBFace network model to perform face and hand detection on the driver image, the DBFace network model needs to be trained.

[0087] It's important to note that the DBFace network model is a face detection network model. Therefore, before using this DBFace network model for hand detection, a category representing hands needs to be added to the detection classification categories before model training and subsequent hand detection. Using the same network model for both face and hand detection in driver images can improve detection speed. Optionally, other network models can be used for hand detection to improve accuracy.

[0088] Optionally, face detection and hand detection can be performed simultaneously on the driver image, or only face detection or only hand detection can be performed on the driver image, depending on the detection requirements. For example, in practical applications, if driver fatigue monitoring, distraction monitoring, and dangerous driving behavior monitoring are required, then face detection and hand detection will be performed on the driver image in step 102. If only driver fatigue monitoring and / or distraction monitoring are required, then only face detection will be performed on the driver image in step 102. If only dangerous driving behavior monitoring is required, then only hand detection will be performed on the driver image in step 102.

[0089] For example, after performing face and hand detection on the driver image, if no face or hand region is detected, the driver image is re-acquired and the detection is performed again, i.e., step 101 and subsequent steps are re-executed.

[0090] Optionally, the face region can be indicated by a face detection bounding box, and the hand region can be indicated by a hand detection bounding box, wherein the face detection bounding box and the hand detection bounding box are square.

[0091] Step 103: Determine the upper body detection area based on the target area.

[0092] In one possible implementation, see Figure 4 Step 103 may specifically include:

[0093] Step 1031: If a face region exists, determine the upper body detection region based on the face region and the preset head-to-body ratio coefficient.

[0094] Step 1032: If there is no face region, then the driver image is used as the upper body detection region.

[0095] Optionally, after using the DBFace network model to perform face and hand detection on the driver image, if both face and hand regions are obtained, or only the face region is obtained, the upper body detection region is determined based on the face region and the preset head-to-body ratio coefficient; if only the hand region is obtained, the driver image is used as the upper body detection region.

[0096] It should be noted that the driver image obtained in step 101 may be in various directions. Correspondingly, the head orientation in the face region in step 1031 may also be in various directions. In order to reduce the impact of the head orientation in the face region on the subsequent determination of the upper body detection area, it is necessary to detect the head orientation in the face region.

[0097] In one possible implementation, when the DBFace network model performs face detection on the driver image, it also performs head orientation detection on the driver image to determine the head orientation.

[0098] Optionally, when performing the DBFace network model to detect the head orientation of the driver image and determine the head orientation, the specific steps may include: based on the multi-head attention mechanism, adding a head orientation detection branch after the feature extraction layer of the DBFace network model to detect the head orientation and determine the head orientation.

[0099] For example, to improve detection speed, a head orientation detection branch is added after the feature extraction layer of the shared network model based on the Multi-head Attention mechanism. This branch performs head orientation detection and classification, enabling the DBFace network model to detect both the driver's face and the driver's head orientation simultaneously. The classification loss function used is the Cross Entropy Loss function.

[0100] A diagram showing the direction of a person's head is shown below. Figure 5 As shown. See also Figure 5 The direction of the head includes facing forward, backward, leftward, and rightward. A forward head direction is defined in the driver's image as the head facing upwards and the angle of the head's orientation relative to the vertical direction being less than a preset angle (see [reference]). Figure 5 (a)); The head facing in the opposite direction is in the driver's image, where the head is facing downwards and the angle of the head's orientation relative to the vertical direction is less than a preset angle (see [reference]). Figure 5 (b)); The head facing left is in the driver's image, where the head faces left and the angle of the head's offset from the horizontal direction is less than a preset angle (see [reference]). Figure 5 (c) in the image; the head facing to the right is when, in the driver's image, the head faces right and the angle of the head's offset from the horizontal direction is less than a preset angle (see [reference]). Figure 5 In (d), the preset angle is 45 degrees. A head offset angle less than the preset angle means that both the clockwise and counterclockwise head offset angles are less than the preset angle.

[0101] In one possible implementation, see Figure 6 Step 1031 may specifically include:

[0102] Step S1: Rotate the face area according to the direction of the head so that the direction of the head in the face area is positive.

[0103] Step S2: Determine the upper body detection area based on the rotated face area and the preset head-to-body ratio coefficient.

[0104] Optionally, in practical applications, if the direction of the head in the face region is determined to be positive, then it is not necessary to rotate the face region; if the direction of the head is determined to be negative, then the direction of the head needs to be rotated 180 degrees, that is, the face region needs to be rotated 180 degrees; if the direction of the head is determined to be left, then the direction of the head needs to be rotated 90 degrees clockwise, that is, the face region needs to be rotated 90 degrees clockwise; if the direction of the head is determined to be right, then the direction of the head needs to be rotated 90 degrees counterclockwise, that is, the face region needs to be rotated 90 degrees counterclockwise, so that the direction of the head in the face region is positive, resulting in the rotated face region.

[0105] For example, the width and height of the upper body detection area are determined based on the width and height of the rotated face region and a preset head-to-body ratio coefficient. According to the Vitruvian man anthropometric proportions, the ratio of head height to body height is ≥ 1:8. In a car cabin, where the driver is seated, the ratio of head height to upper body height is ≥ 1:4. Considering that the driver may stretch, the ratio of head width to upper body width can be ≥ 1:8. In this embodiment, the preset height ratio coefficient is set to 4, and the preset width ratio coefficient is set to 8.

[0106] The formulas for the width and height of the upper body detection area are:

[0107] Width half-body =8 × Width face +α

[0108] Height half-body =4×Height face In the formula +β, Width half-body Height is the width of the upper body detection area. half-body Width is the height of the upper body detection area. face Height is the width of the rotated face region. face The height of the rotated face region is α, and β are compensation values ​​in the preset head-to-body ratio coefficient to make the width and height of the upper body detection region more accurate. Testing revealed that in this embodiment, α = 0.1 × Width is the optimal value. face β = 0.1 × Height face This makes the width and height of the upper body detection area more accurate.

[0109] In another possible implementation, step 1031 may include: determining the upper body detection area based on the direction of the head, according to the face area and a preset head-to-body ratio coefficient.

[0110] For example, instead of rotating the face region based on the head orientation, the upper body detection region is determined based on the head orientation after the head orientation is determined. For instance, if the head orientation is determined to be reversed, the upper body detection region is determined based on the face region and a preset head-to-body ratio coefficient.

[0111] Step 104: Determine the driver's upper body posture based on the upper body detected by the upper body detection area.

[0112] In one possible implementation, see Figure 7 Step 104 may specifically include:

[0113] Step 1041: Use the Yolox-S network model to detect the driver's upper body in the upper body detection area.

[0114] Optionally, in this embodiment, an anchor-free Yolox-S network model is used to detect the upper body region to determine the driver's upper body. Before using the Yolox-S network model to detect the upper body region, the Yolox-S network model needs to be trained.

[0115] Step 1042: When the driver's upper body is detected, the OpenPose network model is used to estimate the pose of the driver's upper body and determine the upper body skeleton points, which include the center point of the target part.

[0116] Optionally, the target body center point includes any one or more of the face center point and hand center point. In this embodiment, the OpenPose network model is used to estimate the driver's upper body pose to determine the upper body skeletal points (e.g., the target body center point) including the target body center point. Figure 8 (As shown). Before using the OpenPose network model to estimate the driver's upper body posture, the OpenPose network model needs to be trained.

[0117] For example, if the driver's upper body is not detected, the driver image is re-acquired and detected, that is, step 101 and subsequent steps are re-executed.

[0118] Step 105: Determine the optimal target area based on the upper body posture and the target area, and detect the target parts of the driver based on the optimal target area.

[0119] Optionally, the target body part may include any one or more of the face and hands.

[0120] In one possible implementation, see Figure 9 Step 105 may specifically include:

[0121] Step 1051: Determine the center point of the face region based on the rotated face region, and determine the center point of the hand region based on the hand region.

[0122] Optionally, the center point of the rotated face region can be taken as the center point of the face region, and the center point of the hand region can be taken as the center point of the hand region.

[0123] Step 1052: Determine the optimal face center point based on the face center point and the face region center point, determine the optimal face region based on the optimal face center point and the face region, and / or, determine the optimal hand center point based on the hand center point and the hand region center point, and determine the optimal hand region based on the optimal hand center point and the hand region.

[0124] In one possible implementation, the average of the coordinates corresponding to the center point of the face and the coordinates corresponding to the center point of the face region can be calculated to determine the optimal center point of the face and the coordinates corresponding to the optimal center point of the face. Then, the optimal face region can be determined based on the coordinates corresponding to the optimal center point of the face and the size of the face detection box corresponding to the face region.

[0125] And / or, calculate the average of the coordinates corresponding to the center point of the hand and the coordinates corresponding to the center point of the hand region to determine the optimal center point of the hand and the coordinates corresponding to the optimal center point of the hand. Then, determine the optimal hand region based on the coordinates corresponding to the optimal center point of the hand and the size of the hand detection box corresponding to the hand region.

[0126] In another possible implementation, a weighted average can be calculated between the coordinates of the face center point and the coordinates of the face region center point to determine the optimal face center point and its corresponding coordinates. The weights can be set according to actual conditions and are not specifically limited here. Then, the optimal face region is determined based on the coordinates of the optimal face center point and the size of the face detection box corresponding to the face region.

[0127] And / or, calculate the weighted average of the coordinates corresponding to the hand center point and the coordinates corresponding to the center point of the hand region to determine the optimal hand center point and its corresponding coordinates. The weights can be set according to the actual situation and are not specifically limited here. Then, determine the optimal hand region based on the coordinates corresponding to the optimal hand center point and the size of the hand detection box corresponding to the hand region.

[0128] Step 1053: Detect the driver's face based on the optimal face region, and / or detect the driver's hands based on the optimal hand region.

[0129] This application embodiment acquires a driver image, performs target part detection on the driver image, and determines the target region. The target part detection includes face detection and hand detection. The target region includes any one or more of the face region and hand region. An upper body detection region is determined based on the target region. The driver's upper body posture is determined based on the upper body detected by the upper body detection region. Based on the upper body posture and the target region, an optimal target region is determined. The driver's target parts are then detected based on the optimal target region. The target parts include any one or more of the face and hands. This can avoid the problem of false detection or inaccurate detection of face or hand and improve the detection accuracy of multi-part detection.

[0130] It should be understood that the sequence number of each step in the above embodiments does not imply the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of this application.

[0131] Figure 10 This is a flowchart illustrating a method for detecting abnormal driving behavior of a driver according to an embodiment of this application. Figure 10 As shown, the method in the embodiments of this application may include:

[0132] Step 201: Determine multiple optimal target areas based on the driver multi-part detection method. The optimal target areas include any one or more of the optimal face area and the optimal hand area.

[0133] Optionally, the above-described driver multi-part detection method can be any of the driver multi-part detection methods provided in any embodiment of this application.

[0134] Step 202: Using the first deep learning model, determine whether the changing trend of the facial contour points included in the multiple optimal facial regions conforms to the fitted sample and whether the duration reaches the preset time. If so, output fatigue driving signal and alarm signal. Calculate the facial pose angle based on the changing trend of the facial contour points included in the multiple optimal facial regions, and determine whether the facial pose angle exceeds the preset angle threshold. If so, output distracted driving signal and alarm signal.

[0135] In one possible implementation, it is determined whether the face in the obtained optimal face region is the driver's face. If it is the driver's face, the next operation is performed on the optimal face region. Otherwise, the driver's image is re-acquired and detected, that is, step 101 and subsequent steps are re-executed.

[0136] For example, based on the first deep learning model, it is determined whether the changing trends of facial contour points in the multiple sets of optimal facial regions conform to the fitted samples and whether the duration reaches a preset time. Specifically, it is determined whether the mouth contour points show an opening trend and conform to the fitted yawning sample, while the duration reaches a first preset time and the stability among the multiple sets of optimal facial regions reaches a first preset threshold. It is also determined whether the eye contour points show a closing trend and conform to the fitted dozing sample, while the duration reaches a second preset time and the stability among the multiple sets of optimal facial regions reaches a second preset threshold. If all the above determinations are correct, a fatigue driving signal and an alarm signal are output.

[0137] Optionally, the facial pose angle is calculated based on the changing trends of facial contour points included in multiple optimal facial regions. Specifically, the facial pose angle includes facial pitch angle and offset angle, as well as gaze pitch angle and offset angle. It is determined whether the above facial pose angle exceeds the corresponding preset angle threshold, and whether the duration reaches a third preset time and the stability among multiple optimal facial regions reaches a third preset threshold. If all the above determinations are true, a distracted driving signal and an alarm signal are output.

[0138] Step 203: Use the second deep learning model to determine whether there is a target object in multiple optimal hand regions, and if there is a target object, determine the category of the target object, and output the dangerous behavior category and alarm signal according to the category of the target object.

[0139] In one possible implementation, it is determined whether the hand gesture in the obtained optimal hand region is compliant. If it is not compliant, the next operation is performed on the optimal hand region. Otherwise, the driver image is re-acquired and detected, that is, step 101 and subsequent steps are re-executed.

[0140] Optionally, based on the second deep learning model, it is determined whether a target object exists in the multiple sets of optimal hand regions, and whether the target object has existed for a period of time greater than a fourth preset time. If both of the above conditions are met, the category of the target object is determined, and a dangerous behavior category and an alarm signal are output according to the category of the target object. The target object can be a mobile device, cigarette, food, or beverage, etc.

[0141] It should be noted that if an exception or exit request is detected during the execution of step 202, the execution of step 202 will be stopped; if an exception or exit request is detected during the execution of step 203, the execution of step 203 will be stopped.

[0142] Optionally, the first deep learning model can be a MobileNetv2 network model, and the second deep learning model can be a YOLOv5 network model. The alarm signal can be at least one of the following: a buzzer alarm signal, a voice alarm signal, an instrument alarm signal, or a seat belt tightening signal, etc., without specific limitations.

[0143] This application embodiment determines multiple optimal target regions by using the driver multi-part detection method provided in any of the foregoing embodiments. The optimal target regions include any one or more of the optimal face region and the optimal hand region. Then, based on a deep learning model, it determines whether the changing trend of facial contour points included in the multiple optimal face regions conforms to the fitted sample and whether the duration reaches a preset time. If it conforms to the fitted sample and the duration reaches the preset time, a fatigue driving signal and an alarm signal are output. The facial pose angle is calculated based on the changing trend of facial contour points included in the multiple optimal face regions, and it is determined whether the facial pose angle exceeds a preset angle threshold. If it exceeds the preset angle threshold, a distracted driving signal and an alarm signal are output. And / or, based on a deep learning model, it determines whether a target object exists in the multiple optimal hand regions, and if a target object exists, its category is determined. Based on the category of the target object, a dangerous behavior category and an alarm signal are output. This avoids the problem of false detection or inaccurate detection of faces or hands, improves the detection accuracy of multi-part detection, and thus improves the detection accuracy of abnormal driving behavior, preventing some traffic accidents caused by poor driving behavior.

[0144] In one possible implementation, step 104 involves detecting the driver's upper body and estimating its posture to determine upper body skeletal points (mainly including facial skeletal points, upper limb skeletal points, and hand skeletal points). This step can also be applied to driver-vehicle voice assistant interaction. In this case, the driver's upper body detection can be based on acquired driver video. For example, after detecting and estimating the driver's upper body posture, confirming that the driver is clapping based on the determined upper body skeletal points can activate the vehicle's voice assistant, thus enabling subsequent interaction.

[0145] In practical applications, when a driver feels fatigued, the in-vehicle voice assistant can be activated after the vehicle stops. The assistant detects the driver's upper body and estimates their posture. Based on the determined upper body skeletal points, it confirms that the driver has selected the fatigue relief option through a specific action. Based on the fatigue relief option, the in-vehicle voice assistant performs another set of specific actions. The assistant then detects the driver's upper body again and estimates their posture. If the determined upper body skeletal points confirm that the driver's actions are consistent with the specific actions of the in-vehicle voice assistant, it can be confirmed that the driver has relieved fatigue and can drive the car normally.

[0146] In another possible implementation, step 104 involves detecting the driver's upper body and estimating its posture to determine upper body skeletal points (mainly including facial skeletal points, upper limb skeletal points, and hand skeletal points). This method can also be applied to in-vehicle motion-sensing games. In this case, the detection of the driver's upper body can be based on acquired driver video. For example, when playing an upper-body motion-sensing game inside a car cabin, after detecting and estimating the driver's upper body posture, the determined upper body skeletal points are used to confirm the driver's limb movements to complete subsequent game operations.

[0147] By detecting and estimating the driver's upper body posture, the upper body skeletal points can be determined and applied to driver-vehicle voice interaction or in-vehicle immersive games, thereby eliminating driver fatigue and improving user experience.

[0148] The following are device embodiments of this application. For details not described in detail, please refer to the corresponding method embodiments described above.

[0149] Figure 11 This is a schematic diagram of the structure of a multi-part detection device for drivers provided in one embodiment of this application. For ease of explanation, only the parts related to the embodiment of this application are shown, and are described in detail below:

[0150] like Figure 11 As shown, the driver multi-part detection device provided in this embodiment may include: an acquisition module 301, a detection module 302, a first determination module 303, and an output module 304.

[0151] The acquisition module 301 is used to acquire the driver's image.

[0152] The detection module 302 is used to detect target parts in the driver image and determine the target region. The target part detection includes face detection and hand detection, and the target region includes any one or more of the face region and the hand region.

[0153] The first determining module 303 is used to determine the upper body detection area based on the target area, and to determine the driver's upper body posture based on the driver's upper body detected through the upper body detection area.

[0154] The output module 304 is used to determine the optimal target area based on the upper body posture and the target area, and to detect the target parts of the driver based on the optimal target area. The target parts include any one or more of the face and hands.

[0155] Optionally, the first determining module 303 is specifically used to: when a face region exists, determine the upper body detection region based on the face region and a preset head-to-body ratio coefficient; when no face region exists, use the driver image as the upper body detection region.

[0156] Optionally, the detection module 302 is specifically used to: use the DBFace network model to detect target parts in the driver image and determine the target area, and to detect the head orientation in the driver image and determine the head orientation.

[0157] Optionally, the first determining module 303 is further specifically used to: rotate the face region according to the direction of the head so that the direction of the head in the face region is positive; and determine the upper body detection region according to the rotated face region and the preset head-to-body ratio coefficient.

[0158] Optionally, the detection module 302 is also specifically used to: based on the multi-head attention mechanism, add a head orientation detection branch after the feature extraction layer of the DBFace network model to perform head orientation detection and determine the head orientation; the head orientation includes forward, backward, leftward and rightward.

[0159] Optionally, the first determining module 303 is further specifically used to: determine the width and height of the upper body detection area based on the width and height of the rotated face area and a preset head-to-body ratio coefficient;

[0160] The width and height of the upper body detection area are:

[0161] Width half-body =8 × Width face +0.1×Width face

[0162] Height nalf-body =4×Height face +0.1×Height face

[0163] Among them, Width half-body Height is the width of the upper body detection area. half-body Width is the height of the upper body detection area. face Height is the width of the rotated face region. face This represents the height of the rotated face region.

[0164] Optionally, the first determining module 303 is further specifically used to: use a Yolox-S network model to detect the driver's upper body in the upper body detection area; when the driver's upper body is detected, use an OpenPose network model to estimate the pose of the driver's upper body and determine the upper body skeleton points; the upper body skeleton points include the center point of the target part, and the center point of the target part includes any one or more of the face center point and the hand center point.

[0165] Optionally, the output module 304 is specifically used for: determining the center point of the face region based on the rotated face region, and determining the center point of the hand region based on the hand region; determining the optimal face center point based on the face center point and the center point of the face region, and determining the optimal face region based on the optimal face center point and the face region; and / or determining the optimal hand center point based on the hand center point and the center point of the hand region, and determining the optimal hand region based on the optimal hand center point and the hand region; detecting the driver's face based on the optimal face region, and / or detecting the driver's hands based on the optimal hand region.

[0166] Optionally, the detection module 302 is also specifically used to: reacquire the driver image and perform detection when no face area or hand area is detected, or when no upper body is detected.

[0167] It should be noted that the information interaction and execution process between the above-mentioned devices / units are based on the same concept as the method embodiments of this application. For details on their specific functions and technical effects, please refer to the method embodiments section, and they will not be repeated here.

[0168] Figure 12 This is a schematic diagram of the structure of a driver abnormal driving behavior detection device provided in one embodiment of this application. Figure 12 As shown, the driver abnormal driving behavior detection device provided in this embodiment may include: a second determination module 401 and an execution module 402.

[0169] The second determining module 401 is used to determine multiple sets of optimal target regions according to the driver multi-part detection method described in any of the above embodiments. The optimal target regions include any one or more of the optimal face region and the optimal hand region.

[0170] The execution module 402 is used to employ a first deep learning model to determine whether the changing trend of facial contour points included in multiple optimal facial regions conforms to the fitted sample and whether the duration reaches a preset time, and outputs a fatigue driving signal and an alarm signal when it conforms to the fitted sample and the duration reaches the preset time; calculates the facial pose angle based on the changing trend of facial contour points included in multiple optimal facial regions, and determines whether the facial pose angle exceeds a preset angle threshold, and outputs a distracted driving signal and an alarm signal when it exceeds the preset angle threshold; and / or, employs a second deep learning model to determine whether there is a target object in multiple optimal hand regions, and determines the category of the target object when it exists; outputs a dangerous behavior category and an alarm signal based on the category of the target object.

[0171] This application also provides a computer program product having program code that, when run in a corresponding processor, controller, computing device, or terminal, executes the steps in any of the above-described driver multi-part detection method embodiments, and / or executes the steps in the above-described driver abnormal driving behavior detection method embodiments, for example... Figure 3 Steps 101 to 105 shown, and / or, as shown Figure 10 Steps 201 to 203 are shown.

[0172] Those skilled in the art will understand that the methods and apparatus proposed in the embodiments of this application can be implemented in various forms, including hardware, software, firmware, dedicated processors, or combinations thereof. Dedicated processors may include application-specific integrated circuits (ASICs), reduced instruction set computers (RISCs), and / or field-programmable gate arrays (FPGAs). The proposed methods and apparatus are preferably implemented as a combination of hardware and software. The software is preferably installed as an application program on a program storage device. This is typically based on a machine with a computer platform, such as one or more central processing units (CPUs), random access memory (RAM), and one or more input / output (I / O) interfaces. An operating system is also typically installed on the computer platform. The various processes and functions described herein may be part of an application program, or a portion thereof may be executed by an operating system.

[0173] Figure 13 This is a schematic diagram of the structure of an electronic control unit provided in an embodiment of this application. Figure 13 As shown, the electronic control unit 10 of this embodiment includes a processor 510 and a memory 520, wherein the memory 520 stores a computer program 521 that can run on the processor 510. When the processor 510 executes the computer program 521, it implements the steps in any of the above method embodiments, for example... Figure 3 Steps 101 to 105 shown, and / or, as shown Figure 10Steps 201 to 203 are shown. Alternatively, when processor 510 executes computer program 521, it implements the functions of each module / unit in the above-described device embodiments, for example... Figure 11 The functions of modules 301 to 304 shown, and / or, as follows: Figure 12 The functions of modules 401 to 402 are shown.

[0174] For example, computer program 521 can be divided into one or more modules / units, one or more of which are stored in memory 520 and executed by processor 510 to complete the solution provided in this application. The one or more modules / units can be a series of computer program instruction segments capable of performing specific functions, which describe the execution process of computer program 521 in electronic control unit 10.

[0175] Those skilled in the art will understand that Figure 13 This is merely an example of an electronic control unit and does not constitute a limitation on the electronic control unit. It may include more or fewer components than shown, or combine certain components, or different components, such as input / output devices, network access devices, buses, etc.

[0176] The processor 510 can be a Central Processing Unit (CPU), or other general-purpose processors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. The general-purpose processor can be a microprocessor or any conventional processor.

[0177] The memory 520 can be an internal storage unit of the terminal device, such as the hard drive or memory of the terminal device, or an external storage device of the terminal device, such as a plug-in hard drive, SmartMedia Card (SMC), Secure Digital (SD) card, or Flash Card. The memory 520 can also include both internal and external storage units of the terminal device. The memory 520 is used to store computer programs and other programs and data required by the terminal device. The memory 520 can also be used to temporarily store data that has been output or will be output.

[0178] Those skilled in the art will clearly understand that, for the sake of convenience and brevity, the above-described division of functional units and modules is merely an example. In practical applications, the above functions can be assigned to different functional units and modules as needed, that is, the internal structure of the device can be divided into different functional units or modules to complete all or part of the functions described above. The functional units and modules in the embodiments can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated unit can be implemented in hardware or as a software functional unit. Furthermore, the specific names of the functional units and modules are only for easy differentiation and are not intended to limit the scope of protection of this application. The specific working process of the units and modules in the above system can be referred to the corresponding process in the foregoing method embodiments, and will not be repeated here.

[0179] In the above embodiments, the descriptions of each embodiment have different focuses. For parts that are not described in detail or recorded in a certain embodiment, please refer to the relevant descriptions of other embodiments.

[0180] Those skilled in the art will recognize that the units and algorithm steps of the various examples described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are implemented in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of this application.

[0181] In the embodiments provided in this application, it should be understood that the disclosed devices / terminal equipment and methods can be implemented in other ways. For example, the device / terminal equipment embodiments described above are merely illustrative. For instance, the division of modules or units is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the displayed or discussed mutual coupling or direct coupling or communication connection may be through some interfaces; the indirect coupling or communication connection between devices or units may be electrical, mechanical, or other forms.

[0182] The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the units can be selected to achieve the purpose of this embodiment according to actual needs.

[0183] Furthermore, the functional units in the various embodiments of this application can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated unit can be implemented in hardware or as a software functional unit.

[0184] If the integrated module / unit is implemented as a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, all or part of the processes in the methods of the above embodiments can also be implemented by a computer program instructing related hardware. The computer program can be stored in a computer-readable storage medium, and when executed by a processor, it can implement the steps of the various method embodiments described above. The computer program includes computer program code, which can be in the form of source code, object code, executable files, or certain intermediate forms. The computer-readable medium can include: any entity or device capable of carrying the computer program code, a recording medium, a USB flash drive, a portable hard drive, a magnetic disk, an optical disk, a computer memory, a read-only memory (ROM), a random access memory (RAM), an electrical carrier signal, a telecommunication signal, and a software distribution medium, etc.

[0185] Figure 14 This is a schematic diagram of the structure of a vehicle provided in one embodiment of this application. Figure 14 As shown, the vehicle 600 in this embodiment includes an electronic control unit 10. The electronic control unit 10 can be the same as the one provided in this application embodiment; its specific functions and technical effects can be found in the method embodiment section, and will not be repeated here.

[0186] Furthermore, the features of the embodiments shown in the accompanying drawings or the various embodiments mentioned in this specification should not be construed as independent embodiments. Rather, each feature described in one example of an embodiment can be combined with one or more other desired features from other embodiments to produce other embodiments not described in words or with reference to the accompanying drawings.

[0187] The above-described embodiments are only used to illustrate the technical solutions of this application, and are not intended to limit them. Although this application has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features. Such modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of this application, and should all be included within the protection scope of this application.

Claims

1. A driver multi-part detection method, characterized by, include: Acquire driver image; The driver image is subjected to target part detection to determine the target region, wherein the target part detection includes face detection and hand detection, and the target region includes any one or more of the face region and the hand region; The upper body detection area is determined based on the target area, and the driver's upper body posture is determined based on the driver's upper body detected through the upper body detection area. Based on the upper body posture and the target area, an optimal target area is determined, and the driver's target parts are detected based on the optimal target area. The target parts include any one or more of the face and hands. The step of determining the optimal target area based on the upper body posture and the target area, and detecting the target parts of the driver based on the optimal target area, includes: The center point of the face region is determined based on the face region, and the center point of the hand region is determined based on the hand region. The optimal face center point is determined based on the face center point and the face region center point, and the optimal face region is determined based on the optimal face center point and the face region. And / or, determine the optimal hand center point based on the hand center point and the center point of the hand region, and determine the optimal hand region based on the optimal hand center point and the hand region; The driver's face is detected based on the optimal face region, and / or the driver's hands are detected based on the optimal hand region.

2. The driver multi-part detection method according to claim 1, characterized in that, The step of determining the upper body detection area based on the target area includes: If a face region exists, the upper body detection region is determined based on the face region and a preset head-to-body ratio coefficient. If no face region exists, the driver image will be used as the upper body detection region.

3. The method for detecting multiple parts of a driver's body according to claim 2, characterized in that, The DBFace network model is used to detect target parts in the driver image to determine the target area, and to detect the head orientation in the driver image to determine the head orientation. The step of determining the upper body detection area based on the face region and a preset head-to-body ratio coefficient includes: The face region is rotated according to the head orientation so that the head orientation in the face region is positive; The upper body detection area is determined based on the rotated face area and the preset head-to-body ratio coefficient.

4. The method for detecting multiple parts of a driver's body according to claim 3, characterized in that, The DBFace network model performs head orientation detection on the driver image to determine the head orientation, including: Based on the multi-head attention mechanism, a head orientation detection branch is added after the feature extraction layer of the DBFace network model to detect and determine the head orientation; the head orientation includes forward, backward, leftward, and rightward.

5. The method for detecting multiple parts of a driver's body according to claim 3, characterized in that, The step of determining the upper body detection area based on the rotated face area and a preset head-to-body ratio coefficient includes: The width and height of the upper body detection area are determined based on the width and height of the rotated face area and a preset head-to-body ratio coefficient. The width and height of the upper body detection area are: in, The width of the upper body detection area. The height of the upper body detection area. The width of the rotated face region. This represents the height of the rotated face region.

6. The method for detecting multiple parts of a driver's body according to claim 3, characterized in that, The determination of the driver's upper body posture based on the driver's upper body detected through the upper body detection area includes: The driver's upper body is detected using a Yolox-S network model in the upper body detection area. When the driver's upper body is detected, the OpenPose network model is used to estimate the pose of the driver's upper body and determine the upper body skeleton points; the upper body skeleton points include the center point of the target part, which includes any one or more of the center point of the face and the center point of the hand.

7. The method for detecting multiple parts of a driver's body according to any one of claims 1 to 6, characterized in that, The method further includes: If the face and hand areas are not detected, or if the upper body is not detected, the driver image is reacquired and detected again.

8. A method for detecting abnormal driving behavior of a driver, characterized in that, include: The driver multi-site detection method according to any one of claims 1 to 7 determines multiple sets of optimal target regions, wherein the optimal target regions include any one or more of the optimal face region and the optimal hand region; Using a first deep learning model, determine whether the changing trend of facial contour points in multiple optimal facial regions matches the fitted sample and whether the duration reaches a preset time. If so, output fatigue driving signal and alarm signal. Calculate facial pose angle based on the changing trend of facial contour points in the multiple optimal facial regions, and determine whether the facial pose angle exceeds a preset angle threshold. If so, output distracted driving signal and alarm signal. And / or, a second deep learning model is used to determine whether a target object exists in multiple optimal hand regions, and if a target object exists, the category of the target object is determined; based on the category of the target object, a dangerous behavior category and an alarm signal are output.

9. A multi-part detection device for drivers, characterized in that, include: The acquisition module is used to acquire images of the driver. The detection module is used to detect target parts in the driver image and determine the target region, wherein the target part detection includes face detection and hand detection, and the target region includes any one or more of the face region and the hand region; The first determining module is used to determine the upper body detection area based on the target area, and to determine the driver's upper body posture based on the driver's upper body detected through the upper body detection area. The output module is used to determine the optimal target area based on the upper body posture and the target area, and to detect the target parts of the driver based on the optimal target area. The target parts include any one or more of the face and hands. The output module is further configured to determine the center point of the face region based on the face region and the center point of the hand region based on the hand region. The optimal face center point is determined based on the face center point and the face region center point, and the optimal face region is determined based on the optimal face center point and the face region. And / or, determine the optimal hand center point based on the hand center point and the center point of the hand region, and determine the optimal hand region based on the optimal hand center point and the hand region; The driver's face is detected based on the optimal face region, and / or the driver's hands are detected based on the optimal hand region.

10. An electronic control unit, comprising a memory and a processor, wherein the memory stores a computer program executable on the processor, characterized in that, When the processor executes the computer program, it implements the steps of the driver multi-site detection method as described in any one of claims 1 to 7, and / or the steps of the driver abnormal driving behavior detection method as described in claim 8.

11. A vehicle, characterized in that, Includes the electronic control unit as described in claim 10.

12. A computer-readable storage medium storing a computer program, characterized in that, When the computer program is executed by the processor, it implements the driver multi-site detection method as described in any one of claims 1 to 7, and / or the driver abnormal driving behavior detection method as described in claim 8.

Images