3D driver distraction detection system based on deep learning and method thereof

Abstract

The provided system and method for deep learning-based 3D driver distraction detection includes steps for generating feature maps and generating 3D keypoint proposals. During model training, a large number of labeled single-frame images (with labeled 3D keypoint coordinates (x, y, z) for each object such as a head, telephone, etc.) are used. The surfaces of the head, telephone, cigarette, etc., can then be reconstructed, and the distances between the head and other objects such as the telephone and cigarette are calculated. Based on those estimated distances, our OMS / DMS system can detect driver distraction behaviors such as telephone calls and smoking.

Description

Technical Field

[0001] The subject matter of this invention generally relates to image processing in driver assistance. More specifically, the subject matter of this invention relates to a deep learning-based 3D driver distraction detection system and method. Background Technology

[0002] As part of the new C-NCAP requirements, GB / T 41797-2022 (a Chinese standard for performance requirements and test methods for driver attention monitoring systems) is about to be implemented. This standard defines telephone conversation activity as maintaining a distance of less than 5 cm between the phone and the driver's face while driving. Original equipment manufacturers (OEMs) in the automotive industry should be required to provide corresponding services to meet this requirement. The standard also specifies the testing procedure for the required distance between the driver's head and the phone.

[0003] However, current classification methods for driver distraction detection on the market neither provide accurate distance detection nor an effective solution that meets the requirements of GB / T 41797-2022.

[0004] In addition, some customers have requested the prediction of the distance between the driver's head and other objects (such as cigarettes), but our current solution using a classifier cannot explicitly detect such smoking activities based on detected distances (such as the distance between the driver's head and the cigarette).

[0005] Therefore, there is a need to design an effective new detection solution that can estimate the 3D shape of human heads and other objects and predict the distances between them to meet these requirements. Summary of the Invention

[0006] The claimed system and method for deep learning-based 3D driver distraction detection provide solutions according to the subject matter of the invention. This deep learning-based 3D detection solution can estimate the 3D shapes of a human head and other objects (such as telephones or cigarettes) and predict the distances between them to meet the requirements for detecting driver distraction.

[0007] In one aspect, a system for detecting driver distraction in 3D based on deep learning is provided. The system includes a detector module comprising: an encoder configured to extract features and vehicle layout information from an input image frame; a decoder configured to generate at least one feature map; and a detection head configured to classify each of at least two objects in the image frame and regress 3D keypoint coordinates of keypoints of each of the at least two objects. The system further includes: a surface reconstruction module configured to reconstruct at least two 3D surfaces for each of the at least two objects; and a computation module configured to calculate the 3D distance between the at least two 3D surfaces.

[0008] On the other hand, a method for detecting driver distraction in 3D based on deep learning is provided. This method includes the following steps: extracting features and vehicle layout information of an input image frame via an encoder of a detector module; generating at least one feature map via a decoder of the detector module; and classifying each of at least two objects in the image frame via a detector head of the detector module, and regressing the 3D keypoint coordinates of keypoints of each of the at least two objects. The method further includes the following steps: reconstructing at least two 3D surfaces for each of the at least two objects via a surface reconstruction module; and calculating the 3D distance between the at least two 3D surfaces via a computation module.

[0009] In another aspect, a computer-readable medium is provided for storing instructions that, when executed by one or more processors, can perform steps for a method of deep learning-based 3D detection of driver distraction. Attached Figure Description

[0010] The subject matter of the invention can be better understood by referring to the following description of non-limiting embodiments. In the drawings, the same reference numerals refer to corresponding parts, as shown in the following figures:

[0011] Figure 1 An exemplary structural diagram of a system for detecting driver distraction in 3D based on deep learning, according to one or more embodiments of the subject matter of the present invention, is shown.

[0012] Figure 2 An exemplary diagram of a detector module for a deep learning-based 3D driver distraction detection system according to one or more embodiments of the subject matter of the present invention is shown;

[0013] Figure 3An exemplary flowchart of a method for deep learning-based 3D driver distraction detection according to one or more embodiments of the subject matter of the present invention is shown; and

[0014] Figure 4 An exemplary process diagram is shown for a method for detecting driver distraction in 3D based on deep learning, according to one or more embodiments of the subject matter of the present invention. Detailed Implementation

[0015] The following provides a detailed description of one or more embodiments of the subject matter of this invention; however, it should be understood that the disclosed embodiments are merely examples of the subject matter of this invention that may be embodied in various alternative forms. The drawings are not necessarily drawn to scale; some features may be enlarged or minimized to show details of specific components. Therefore, the specific structural and functional details disclosed herein should not be construed as limiting, but rather serve only as a representative basis for teaching those skilled in the art to employ the subject matter of this invention in different ways.

[0016] The 3D key point detection solution provided in this invention can estimate the 3D shape of a person's head and other items such as a telephone or cigarette in a vehicle compartment, and predict the distance between the head and the item to meet the requirements of GB / T41797-2022.

[0017] In one aspect, the subject matter of the present invention provides a system for 3D detection of driver distraction based on deep learning.

[0018] Figure 1 An exemplary structural diagram 100 of a system for detecting driver distraction in 3D based on deep learning, according to one or more embodiments of the subject matter of the present invention, is shown.

[0019] like Figure 1 As shown, the provided system includes a detector module 110, a reconstruction module 120, and a computation module 130. The detector module 110 can be in the form of a neural network model. During the model training phase, a large number of labeled single-frame images (with labeled 3D keypoint coordinates (x, y, z) for each object such as a head, telephone, cigarette, etc.) are used to train the neural network model in the detector module 110. The trained detector module 110 can output predicted 3D keypoints for a human face, telephone, or cigarette. These identified 3D keypoints can then be input into the reconstruction module 120, where the 3D curved surfaces of the human head, telephone, or cigarette can be reconstructed, and thus the distance between the human head and other objects such as telephones or cigarettes can be calculated in the computation module 130. Based on those estimated distances, an OMS / DMS system in a car can detect driver distraction behaviors such as telephone calls and smoking.

[0020] Figure 2An exemplary figure 200 illustrates a detector module of a deep learning-based 3D driver distraction detection system according to one or more embodiments of the subject matter of the present invention. The detector module (which may be...) Figure 1 The pipeline of the detector module 110 shown includes three parts: an encoder 210 for extracting image features and carriage layout information 230; a decoder 240 for generating at least one feature map 250 for 3D object detection; and a detection head for classifying each object and correspondingly regressing the 3D keypoint coordinates of their keypoints into feature vectors 260 and 270.

[0021] like Figure 2 As shown in the main components of the detector module, firstly, the input to encoder 210 includes vehicle compartment layout information 230, which may include compartment size information (such as the length, width, and height of the vehicle compartment interior), seat position layout, and camera positioning within the compartment. Driver behavior can be captured by cameras positioned within the compartment (e.g., for a driver monitoring system (DMS)). That is, the DMS camera can capture image frames 220 of the compartment interior, which, along with the compartment layout information 230, can be input to encoder 210 of the detector module.

[0022] Since the detector module is a trained neural network model, it can classify and regress a point based on the objectivity probability score and classification probability score of the point in the input image frame 230.

[0023] As mentioned above, encoder 210 can extract image features from input image frame 220. Decoder 240 can then generate and output at least one feature map 250 for 3D keypoint detection. In this feature map, each point in the input image frame can be described by an objective probability score obj, a classification probability score cls, and its 3D coordinates (x, y, z). In the example, for a point in input image frame 230, the probability that the point belongs to a certain object can first be determined. For points whose objective probability score is higher than a threshold, the probability that the point belongs to any certain category can then be determined. Depending on its classification probability score, the point could belong to a human head, or to a mobile phone or cigarette, etc. Next, for points in the input image frame where both the objective probability score and the classification probability score are higher than a set threshold, the coordinates (x, y, z) of these 3D keypoints are collected along with the objective probability score and the classification probability score to generate a feature map. The input image frame can generate at least one feature map for 3D object detection.

[0024] For example, the values ​​of the objectivity probability score and the classification probability score can both range from 0 to 1, with a threshold of 0.25 set. For a point in the input image frame, if its objectivity probability score and classification probability score are both higher than 0.25, then the 3D keypoint coordinates of that keypoint will be regressed to the corresponding feature vector.

[0025] An exemplary feature vector for 3D point detection extracted from at least one feature map can be as follows: Figure 2 As shown on the right side. The upper vector 260 is formed by those points extracted from at least one feature map belonging to the human head category. In this example, the 3D keypoint coordinates (x, y, z) of those points with an objectivity probability score obj greater than 0.25 and a human head classification probability score cls greater than 0.25 can be regressed into the upper vector 260 for human heads. Similarly, the lower vector 270 is formed by those points extracted from at least one feature map belonging to the telephone (or cigarette) category. In this example, the 3D keypoint coordinates (x, y, z) of those points with an objectivity probability score greater than 0.25 and a telephone (or cigarette) classification probability score greater than 0.25 can be regressed into the lower vector 270 for telephone (or cigarette). The 3D keypoint coordinates (x, y, z) can be filled into the slots following the corresponding head in the corresponding feature vector. Alternatively, the value of the objectivity probability score can be different from the value of the classification probability score. It is conceivable that the existence, classification, and 3D keypoint coordinates of other objects can also be included in at least one feature map, and their feature vectors can be extracted.

[0026] All regression values ​​(x, y, z) in each positive eigenvector can be used to construct 3D keypoints for a human head, telephone, etc., in the desired order. In subsequent processing, the 3D keypoints regressed to the human head can be used to reconstruct the 3D curved surface of a portion of the human head, and the face, mouth, eyes, etc., can be reconstructed on this surface. Similarly, the 3D keypoints regressed to the telephone (or cigarette) can be used to reconstruct the 3D surface of the telephone (or cigarette). Therefore, after obtaining the 3D keypoint coordinates (x, y, z) output from the detector module 110, the 3D surface of each object can be reconstructed. In the reconstruction module 120, the 3D surfaces of the human mouth and other objects such as the telephone and cigarette can be reconstructed separately. For surface reconstruction, a mature field, there are many potentially robust algorithms to choose from. For example, Power Crust, Poisson, and other surface reconstruction algorithms would be suitable surface reconstruction algorithms, and finally, the distance from the driver's head to the telephone (or cigarette) required for prediction can be calculated in the computation module 130.

[0027] In another aspect, the subject of this invention provides a method for 3D detection of driver distraction based on deep learning.

[0028] Figure 3 An exemplary flowchart of a method for detecting driver distraction in 3D based on deep learning, according to one or more embodiments of the subject matter of the present invention, is shown.

[0029] First, the detector module adopts a neural network model, such as... Figure 1 The detector module 110 is shown. The neural network training process includes feeding the neural network paired vehicle compartment layout information and input image frames. Vehicle compartment layout information can be obtained from factory information set for vehicle type, and different vehicle types can have different vehicle compartment layout information, such as vehicle compartment size or dimensions (including the length, width, height, etc. of the interior space), the location of interior facilities (such as front and rear seats), and the location of DMS cameras in the vehicle compartment. For vehicles of the same type, the vehicle compartment layout information can be fixed. Since different types can have different vehicle compartment layout or size parameters, the neural network model will ultimately predict 3D key points based on the layout and size information of the vehicle compartment. Therefore, the neural network model should be trained according to the vehicle type. For each known vehicle model among various known vehicle models, the detector module only needs to be trained once, and the neural network model of the detector module does not need to be retrained when the system is to be used in a new known vehicle type. Only those newly designed vehicles of unknown type need to undergo the training step. Therefore, in step S310, the vehicle type should be determined first.

[0030] For the training process of the neural network model for unknown vehicle types, in step S320, a large number of labeled single-frame images (with labeled 3D keypoint coordinates (x, y, z) for each object such as a head, phone, cigarette, etc.) are used to train the neural network model of the detector module. The neural network model outputs predicted 3D keypoints for a human face, phone, or cigarette. Then, a loss function can be used to supervise the model to learn how to predict 3D keypoints close to the true values, and the neural network model will ultimately predict 3D keypoints based on the layout information of the vehicle body type.

[0031] For vehicle types already known to the neural network model, in step S330, the detector module can obtain input of the carriage layout information and input image frames captured from the DMS camera. The encoder of the detector module can extract features from the input image frames. Different types of vehicle carriages can have multiple arrangements, which have various carriage layout information; this is why carriage layout information needs to be used as an input parameter of the system.

[0032] Based on this information, in step S340, the decoder of the detector module can predict 3D key points of the human face and the phone to generate at least one feature map. To infer 3D key points in a newly known type of car compartment, the compartment layout parameters can be changed, but it will not be necessary to modify only one input image frame captured by the camera.

[0033] Next, in step S350, the decoder head of the detector module can classify each object and regress the 3D key point coordinates of its key points.

[0034] After obtaining the 3D keypoint output from the detector module, in step S360, the 3D surface can be reconstructed. All (x, y, z) values ​​in each positive feature vector are used to construct 3D keypoints for the human head, telephone, etc., in a predetermined order. From those 3D keypoints of the human head, we can obtain the 3D keypoints of the mouth, face, etc. The surface reconstruction process can be performed by, for example, PowerCrust, Poisson, and / or other surface reconstruction algorithms.

[0035] Therefore, in step S360, the 3D surfaces of the human mouth and other objects such as telephones and cigarettes can be reconstructed separately, and the required distance for prediction can then be calculated. As an example, a driver distraction alarm can be generated when the calculated 3D distance is shorter than a threshold (such as 5 cm set for GB / T41797-2022).

[0036] Figure 4 An exemplary process diagram 400 illustrates a method for detecting driver distraction in 3D based on deep learning, according to one or more embodiments of the subject matter of the present invention.

[0037] exist Figure 4 On the left side, input image frame 410 captured by the DMS camera in the carriage has been input into detector module 420 in a system for detecting driver distraction based on deep learning, as provided in the subject matter of the invention. Figure 4 As schematically shown on the left, this image frame captures the driver's head and his phone inside the carriage. Additionally, detector module 420 can also obtain chassis layout information, for example, from factory setup information, as previously described. It is conceivable that the image of the interior facilities of the carriage captured in input image frame 410 can correspond to the obtained chassis layout information for that carriage. Figure 4As shown, at least the coordinates of the driver's seat in the input image frame 410 can correspond to the cabin layout information in the reference camera coordinate system. After processing by the detection module 420 in the form of a neural network model, 16 key points along the facial contour 422 on the head, 14 key points at the mouth 424 on the head, 8 key points on the nose 426, 6 key points on each of his two eyes 428, and 7 key points on each of his two eyebrows 430 are detected from the input image frame 410. Therefore, the neural network model can regress these 64 key points on the driver's head and can reconstruct the 3D key point coordinates (x0, y0, z0)...(x 63 , y 63 , z 63 Extracted to, for example Figure 2 The head vector 260 shown can reconstruct the 3D curved surface 440 of the driver's head, including the driver's face, mouth, nose, eyes, and eyebrows, as can be seen in... Figure 4 This can be seen from the text.

[0038] Additionally, key points such as those of a mobile phone can be detected from image frame 410. For example, a mobile phone can be viewed as a hexahedron, and the eight vertices 450 of this hexahedron can be considered as 3D key points of the phone. Therefore, the neural network model can classify these eight key points on the phone and can regress the 3D key point coordinates (x0, y0, z0)…(x7, y7, z7) of these eight key points to a coordinate system such as... Figure 2 The telephone vector 270 is shown. This telephone vector can reconstruct the 3D surface 460 of the telephone, as shown. Figure 4 As shown.

[0039] Therefore, the 3D distance d, 470 between the driver's head and the phone can be calculated to detect driver distraction. Figure 4 As shown in the example, 3D distance can be detected in real time or at certain time intervals. When the detected distance is shorter than the minimum distance set at 5 cm according to GB / T 41797-2022, a driver distraction alert will be generated to remind the driver to focus on driving.

[0040] This invention is designed to address the problem of manual distraction detection in ADAS Space 2. It provides a novel 3D keypoint detection head for estimating the 3D contours of the driver's head and a telephone. This 3D keypoint detection head can provide a quantitative prediction of the distance between the head and the telephone, as required by GB / T 41797-2022. No such solution exists in the industry for 3D distance detection. Furthermore, this invention can be based on a 3D object detection pipeline, a lightweight model of 3D object segmentation. The method can also be extended to detect smoking behavior by the distance between a cigarette and the driver's face. Generally, this invention can improve the accuracy and interpretability of our manual distraction detection model. Moreover, the method can be extended to 3D perception tasks in L2 / L3 / L4 autonomous driving scenarios, such as obstacle detection and pedestrian detection. Compared to segmentation-based methods, this invention is based on 3D keypoint detection, which can be more lightweight and faster. Compared to other 3D object detection methods that only detect 3D bounding boxes, the subject matter of this invention can detect the detailed 3D shape of an object, which is more precise and accurate. Therefore, the subject matter of this invention can achieve the above-mentioned advantages in ADAS / autonomous driving applications used in our daily lives.

[0041] The methods provided in one or more embodiments of the subject matter of this invention can be performed using any combination of one or more computer-readable media. A computer-readable medium can be a computer-readable signal medium or a computer-readable storage medium. A computer-readable storage medium can be, for example, but not limited to, electronic, magnetic, optical, electromagnetic, infrared, or semiconductor systems, devices, or apparatuses, or any suitable combination of the foregoing media. More specific examples (not an exhaustive list) of computer-readable storage media may include, for example: an electrical connection having one or more wires, a portable computer floppy disk, a hard disk, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fiber, portable optical disc read-only memory (CD-ROM), optical storage device, magnetic storage device, or any suitable combination of the foregoing media. In the context of this disclosure, a computer-readable storage medium can be any tangible medium that can include or store a program for use by or in connection with an instruction execution system, device, or apparatus.

[0042] As used in this disclosure, elements or steps listed in the singular followed by the word "a / an" should be understood to not exclude a plurality of said elements or steps unless such exclusion is specifically stated. Furthermore, references to "embodiments" or "examples" in this disclosure are not intended to be exclusive and include the existence of other embodiments of the described features. The terms "first," "second," "third," etc., are used for identification only and are not intended to emphasize numerical requirements or order of placement of their objects.

[0043] The references to methods for reproducing a constant sound field in the subject matter of this invention include the following:

[0044] Project 1: In one or more embodiments, the subject matter of this invention provides a system for deep learning-based 3D driver distraction detection, comprising:

[0045] The detector module includes:

[0046] The encoder is configured to extract features and carriage layout information from the input image frame;

[0047] Decoder, the decoder being configured to generate at least one feature map; and

[0048] A detection head configured to classify each of at least two objects in the image frame and regress the 3D keypoint coordinates of keypoints of each of the at least two objects.

[0049] A surface reconstruction module, configured to reconstruct at least two 3D surfaces for each of the at least two objects, and

[0050] A calculation module configured to calculate the 3D distance between the at least two 3D surfaces.

[0051] Project 2. The system as described in Project 1, wherein the detector module may be in the form of a neural network model, which can be trained using a large amount of paired carriage layout information and carriage image frames known for various types of vehicles.

[0052] Project 3. The system as described in Project 1 or 2, wherein the carriage layout information may include carriage dimensions, seat layout, and camera positioning within the carriage.

[0053] Item 4. The system of any one of Items 1 to 3, wherein the input image frame can be captured from the camera arranged in the carriage.

[0054] Project 5. The system of any one of Projects 1 to 4, wherein the at least one feature map comprises at least two feature vectors, each feature vector having an objectivity probability score, a classification probability score, and at least one regressed 3D keypoint coordinate of the keypoint of each of the at least two objects.

[0055] Item 6. The system of any one of items 1 to 5, wherein the at least two objects may include a human head and another item, and wherein the other item may include a telephone or a cigarette.

[0056] Item 7. The system as described in any one of Items 1 to 6, wherein the number of 3D keypoints from one of the at least two objects may be different from the number of 3D keypoints from the other of the at least two objects.

[0057] Item 8. The system of any one of items 1 to 7, wherein the number of the 3D key points from the human head may be greater than the number of the 3D key points from the other article.

[0058] Item 9. The system of any one of Items 1 to 8, wherein the surface reconstruction module configured to reconstruct at least two 3D surfaces for the at least two objects may use algorithms including Power Crust or Poisson.

[0059] Item 10. The system as described in any one of Items 1 to 9, wherein a driver distraction alert can be generated when the calculated 3D distance is shorter than a threshold.

[0060] Project 11: In one or more embodiments, the subject matter of the present invention further provides a method for deep learning-based 3D driver distraction detection, comprising the following steps:

[0061] The encoder of the detector module extracts features and carriage layout information from the input image frame;

[0062] At least one feature map is generated via the decoder of the detector module;

[0063] The detector module classifies each of at least two objects in the image frame via its detection head, and regresses the 3D key point coordinates of the key points of each of the at least two objects.

[0064] At least two 3D surfaces are reconstructed for each of the at least two objects via the surface reconstruction module; and

[0065] The 3D distance between the at least two 3D surfaces is calculated via a computing module.

[0066] Project 12. The device of the method as described in Project 11, wherein the detector module may be in the form of a neural network model, which can be trained using a large amount of paired carriage layout information and carriage image frames known for various types of vehicles.

[0067] Item 13. The method device as described in Item 11 or 12, wherein the carriage layout information may include carriage dimensions, seat layout, and camera positioning in the carriage.

[0068] Item 14. The method apparatus of any one of items 11 to 13, wherein the input image frame can be captured from the camera arranged in the carriage.

[0069] Item 15. The method apparatus of any one of Items 11 to 14, wherein the at least one feature map comprises at least two feature vectors, each feature vector having an objectivity probability score, a classification probability score, and at least one regressed 3D keypoint coordinate of the keypoint of each of the at least two objects.

[0070] Item 16. The method apparatus of any one of items 11 to 15, wherein the at least two objects may include a human head and another item, and wherein the other item may include a telephone or a cigarette.

[0071] Item 17. The method apparatus of any one of items 11 to 16, wherein the number of the 3D keypoints from one of the at least two objects may be different from the number of the 3D keypoints from the other of the at least two objects.

[0072] Item 18. The method apparatus of any one of items 11 to 17, wherein the number of the 3D key points from the human head may be greater than the number of the 3D key points from the other article.

[0073] Item 19. The method apparatus of any one of Items 11 to 18, wherein reconstructing at least two 3D surfaces for each of the at least two objects may use algorithms including Power Crust or Poisson.

[0074] Item 20. The method apparatus of any one of items 11 to 19, further comprising generating a driver distraction alarm when the calculated 3D distance is shorter than a threshold.

[0075] Item 21. In one or more embodiments, the subject matter of the invention further provides a computer-readable medium storing instructions that, when executed by one or more processors, can perform the steps of a method for detecting driver distraction in 3D based on deep learning according to any one of items 11 to 20.

Claims

1. A system for detecting driver distraction in 3D based on deep learning, comprising: The detector module includes: The encoder is configured to extract features and carriage layout information from the input image frame; Decoder, the decoder being configured to generate at least one feature map; and A detection head configured to classify each of at least two objects in the image frame and regress the 3D keypoint coordinates of keypoints of each of the at least two objects. A surface reconstruction module, configured to reconstruct at least two 3D surfaces for each of the at least two objects, and A calculation module configured to calculate the 3D distance between the at least two 3D surfaces.

2. The system of claim 1, wherein the detector module may be in the form of a neural network model, which can be trained using a large number of paired carriage layout information and carriage image frames known for various types of vehicles.

3. The system as described in claim 1, wherein the carriage layout information may include carriage size, seat positioning, and the positioning of cameras in the carriage.

4. The system of claim 2, wherein the input image frame can be captured from the camera arranged in the carriage.

5. The system of claim 1, wherein the at least one feature map comprises at least two feature vectors, each feature vector having an objectivity probability score, a classification probability score, and at least one regressed 3D keypoint coordinate of the keypoint of each of the at least two objects.

6. The system of claim 5, wherein the at least two objects may include a human head and another item, and wherein the other item may include a telephone or a cigarette.

7. The system of claim 5, wherein the number of 3D keypoints from one of the at least two objects may be different from the number of 3D keypoints from the other of the at least two objects.

8. The system of claim 7, wherein the number of 3D keypoints from the human head may be greater than the number of 3D keypoints from the other object.

9. The system of claim 1, wherein the surface reconstruction module configured to reconstruct at least two 3D surfaces for the at least two objects may use algorithms including Power Crust or Poisson.

10. The system of claim 1, wherein a driver distraction alert can be generated when the calculated 3D distance is shorter than a threshold.

11. A method for detecting driver distraction in 3D based on deep learning, comprising the following steps: The encoder of the detector module extracts features and carriage layout information from the input image frame; At least one feature map is generated via the decoder of the detector module; The detector module classifies each of at least two objects in the image frame via its detection head, and regresses the 3D key point coordinates of the key points of each of the at least two objects. At least two 3D surfaces are reconstructed for each of the at least two objects using the surface reconstruction module; as well as The 3D distance between the at least two 3D surfaces is calculated via a computing module.

12. The method of claim 11, wherein the detector module may be in the form of a neural network model, which can be trained using a large number of paired carriage layout information and carriage image frames known for various types of vehicles.

13. The method of claim 11, wherein the carriage layout information may include carriage size, seat positioning, and camera positioning in the carriage.

14. The method of claim 12, wherein the input image frame can be captured from the camera arranged in the carriage.

15. The method of claim 11, wherein the at least one feature map comprises at least two feature vectors, each feature vector having an objectivity probability score, a classification probability score, and at least one regressed 3D keypoint coordinate of the keypoint of each of the at least two objects.

16. The method of claim 15, wherein the at least two objects may include a human head and another item, and wherein the other item may include a telephone or a cigarette.

17. The method of claim 15, wherein the number of 3D keypoints from one of the at least two objects may be different from the number of 3D keypoints from the other of the at least two objects.

18. The method of claim 17, wherein the number of 3D keypoints from the human head may be greater than the number of 3D keypoints from the other article.

19. The method of claim 11, wherein reconstructing at least two 3D surfaces for each of the at least two objects may use algorithms including Power Crust or Poisson.

20. The method of claim 11, further comprising generating a driver distraction alert when the calculated 3D distance is shorter than a threshold.

21. A computer-readable medium storing instructions that, when executed by one or more processors, can perform the steps of the method according to any one of claims 11 to 20.

Images