Vehicle occupant head posture recognition method, vehicle, medium and product

Abstract

The application discloses a vehicle occupant head posture recognition method, a vehicle, a computer readable storage medium and a computer program product. The method comprises: acquiring a three-dimensional depth image of a target occupant head; performing feature extraction processing on the three-dimensional depth image to determine head feature data; in the case where any feature point in the head feature data is detected to be in a temporary shielding state, constructing a target head three-dimensional model of the target occupant according to historical feature points corresponding to the missing feature point, the head feature data and the three-dimensional depth image; and recognizing a target head posture of the target occupant according to the target head three-dimensional model. In this way, in the temporary shielding state, the historical feature points can be called to combine the head feature data and the three-dimensional depth image to construct a complete and accurate target head three-dimensional model, and the head posture of the target occupant is recognized based on the model, the accuracy of the head posture recognition is improved, the anti-shielding capability is improved, and accurate decision basis is provided for subsequent vehicle-mounted functions.

Description

Technical Field

[0001] This application relates to the field of vehicle technology, and in particular to a method for recognizing the head posture of occupants in a vehicle, a vehicle, a computer-readable storage medium, and a computer program product. Background Technology

[0002] Recognizing the head posture of occupants inside a vehicle can provide data for functions such as safe driving and in-cabin entertainment. However, components such as the steering wheel and sun visors inside the vehicle cabin, as well as occupant movements, can easily obstruct the occupant's head, making it impossible to accurately identify the occupant's head posture. Summary of the Invention

[0003] This application provides a method for recognizing the head posture of occupants in a vehicle, a vehicle, a computer-readable storage medium, and a computer program product.

[0004] This application provides a method for recognizing the head posture of a vehicle occupant, the method comprising: A three-dimensional depth image of the head of the target occupant is acquired, wherein the three-dimensional depth image is acquired by the three-dimensional imaging device of the vehicle; The three-dimensional depth image is subjected to feature extraction processing to determine head feature data; If any feature point in the head feature data is detected to be in a temporary occlusion state, a three-dimensional model of the target occupant's head is constructed based on the historical feature points corresponding to the missing feature points in the temporary occlusion state, the head feature data, and the three-dimensional depth image. Based on the three-dimensional model of the target head, the target occupant's target head posture is identified.

[0005] This process involves acquiring a 3D depth image of the target occupant's head; performing feature extraction on the 3D depth image to determine head feature data; and constructing a 3D head model of the target occupant based on historical feature points corresponding to the missing feature points, the head feature data, and the 3D depth image when any feature point in the head feature data is temporarily occluded. Finally, the target head pose of the target occupant is identified based on this 3D head model. By acquiring multiple consecutive frames of 3D depth images of the target occupant's head, feature extraction can be performed to obtain head feature data, and feature points within this data can be detected. Even under temporary occlusion, historical feature points can be retrieved. By combining the head feature data and the 3D depth image, the head contour details of the target occupant can be accurately reconstructed, constructing a complete and accurate 3D head model. Based on this 3D head model, the head pose of the target occupant can be identified, improving the accuracy and anti-occlusion capability of head pose recognition and providing precise decision-making support for subsequent in-vehicle functions.

[0006] In some embodiments, the vehicle cabin is equipped with at least one three-dimensional imaging device; and / or The three-dimensional imaging device is located at the left front of the vehicle cabin; and / or The three-dimensional imaging device is installed on the left front ceiling of the vehicle cabin; and / or The three-dimensional imaging device is installed on the left front A-pillar of the vehicle cabin; and / or The three-dimensional imaging device is installed on the left B-pillar of the vehicle cabin; and / or The three-dimensional imaging device is located at the right front of the vehicle cabin; and / or The three-dimensional imaging device is installed on the right front ceiling of the vehicle cabin; and / or The three-dimensional imaging device is installed on the right front A-pillar of the vehicle cabin; and / or The three-dimensional imaging device is installed on the right-side B-pillar of the vehicle cabin; and / or The field of view of the three-dimensional imaging device is a preset field of view, which is 80°-100°; and / or The optical axis of the lens of the three-dimensional imaging device is at a preset angle to the axial direction directly in front of the vehicle cabin; the preset angle is 30°-70°; and / or The center point of the lens of the three-dimensional imaging device is at a preset vertical distance from the seat cushion plane of the vehicle cabin, the preset vertical distance being 0.5 meters to 0.7 meters; and / or The center point of the lens of the three-dimensional imaging device is at a preset horizontal distance from the center line of the shoulder of the vehicle cabin, and the preset horizontal distance is 0.4 meters to 0.6 meters.

[0007] Thus, at least one 3D imaging device is installed in the vehicle cabin; the 3D imaging device is located at the left front of the vehicle cabin; the 3D imaging device is located at the left front of the vehicle cabin roof; the 3D imaging device is located at the left front of the vehicle cabin A-pillar; the 3D imaging device is located at the left side of the vehicle cabin B-pillar; the 3D imaging device is located at the right front of the vehicle cabin; the 3D imaging device is located at the right front of the vehicle cabin roof; the 3D imaging device is located at the right front of the vehicle cabin A-pillar; the 3D imaging device is located at the right side of the vehicle cabin B-pillar; the field of view of the 3D imaging device is a preset field of view, which is 80°-100°; the optical axis of the lens of the 3D imaging device is at a preset angle to the front axis of the vehicle cabin, which is 30°-70°; the center point of the lens of the 3D imaging device is at a preset vertical distance from the seat cushion plane of the vehicle cabin, which is 0.5 meters-0.7 meters; the center point of the lens of the 3D imaging device is at a preset horizontal distance from the shoulder centerline of the vehicle cabin, which is 0.4 meters-0.6 meters. In this way, by setting the configuration, position and parameters of the 3D imaging equipment, interference factors of frontal placement can be avoided, ensuring that accurate and reliable 3D depth images can be acquired in complex in-vehicle environments, laying a reliable foundation for subsequent feature processing and the establishment of a 3D model of the target head.

[0008] In some embodiments, the head feature data includes reference feature points and symmetrical feature points; the reference feature points include the tip of the nose, the root of the nose, and the apex of the chin; the symmetrical feature points include the tragus, the auricle, the vertex of the head, the alar of the nose, the angle of the mandible, the submental point, the canthus of the eye, the orbital point, and / or the occipital protuberance.

[0009] Thus, the head feature data includes baseline feature points and symmetrical feature points. Baseline feature points include the tip of the nose, the root of the nose, and the chin. Symmetrical feature points include the tragus, the auricle, the vertex of the head, the ala of the nose, the angle of the mandible, the submental point, the canthus of the eye, the orbital point, and / or the occipital protuberance. By clearly defining the baseline feature points, a foundation can be laid for constructing a symmetrical reference plane. Simultaneously, by clearly defining the symmetrical feature points, it can be ensured that each region of the head has a corresponding symmetrical mapping basis, providing accurate data for subsequently establishing a 3D model of the target head.

[0010] In some implementations, the symmetrical feature points include the missing feature points on the first side and first feature points on the first side excluding the missing feature points. When it is detected that any feature point in the head feature data is in a temporary occlusion state, constructing a target head 3D model of the target occupant based on the historical feature points corresponding to the missing feature points in the temporary occlusion state, the head feature data, and the 3D depth image includes: Based on the aforementioned reference feature points, determine the symmetrical reference plane; If the missing feature points are detected to be missing continuously within a preset time, the historical feature points are symmetrically mapped based on the symmetrical reference plane to determine the first predicted feature point in the second side that is symmetrical to the historical feature points, wherein the second side is symmetrical to the first side. Based on preset constraints, a three-dimensional model of the target head is constructed according to the first feature point, the first predicted feature point, the historical feature point, and the three-dimensional depth image.

[0011] Thus, a symmetrical reference plane is determined based on the baseline feature points. When missing feature points are detected to be continuously missing within a preset time period, symmetrical mapping is performed on historical feature points based on the symmetrical reference plane to determine the first predicted feature point symmetrical to the historical feature points on the second side, where the second side is symmetrical to the first side. Based on preset constraints, a 3D model of the target head is constructed using the first feature point, historical feature points, the first predicted feature point, and the 3D depth image. In this way, by constructing a symmetrical reference plane using baseline feature points, symmetrical mapping can be performed on historical feature points corresponding to continuously missing feature points within a preset time period to generate the first predicted feature point. Then, combining the first feature point, historical feature points, the 3D depth image, and preset constraints, a complete 3D model of the target head is constructed, solving the problems of inaccurate feature point completion and model reconstruction distortion under occlusion conditions, and improving the accuracy of head pose recognition based on the 3D model of the target head.

[0012] In some implementations, constructing the three-dimensional model of the target head based on preset constraints, using the first feature point, the first predicted feature point, the historical feature point, and the three-dimensional depth image, includes: Based on the symmetric reference plane, the first feature point is subjected to symmetric mapping processing to determine the second predicted feature point on the second side that is symmetric to the first feature point; Based on preset constraints, a three-dimensional model of the target head is constructed according to the first feature point, the first predicted feature point, the historical feature point, the second predicted feature point, and the three-dimensional depth image.

[0013] Thus, based on a symmetrical reference plane, the first feature point is symmetrically mapped to determine a second predicted feature point on the second side that is symmetrical to the first feature point. Based on preset constraints, a 3D model of the target head is constructed using the first feature point, historical feature points, first predicted feature points, second predicted feature points, and a 3D depth image. In this way, by performing symmetrical mapping on the unoccluded first feature point on the first side, the second predicted feature point can be determined to complete the feature information of the second side. Furthermore, by fusing the first feature point, historical feature points, first predicted feature points, second predicted feature points, and the 3D depth image, a 3D model of the target head is constructed under preset constraints, ensuring the integrity and accuracy of the 3D model and thus improving the accuracy of head pose recognition based on the 3D model of the target head to a certain extent.

[0014] In some embodiments, the method further includes: The target prediction weight is determined based on the number of the symmetrical feature points; If the missing feature points are detected to be missing continuously within the preset time period, the target head three-dimensional model is constructed based on the preset constraints, the target prediction weight, the first feature point, the historical feature point, the first predicted feature point, the second predicted feature point, and the three-dimensional depth image.

[0015] Thus, the target prediction weight is determined based on the number of symmetrical feature points. If missing feature points are detected to be continuously missing within a preset time period, a 3D model of the target head is constructed based on preset constraints, the target prediction weight, the first feature point, historical feature points, the first predicted feature point, the second predicted feature point, and the 3D depth image. In this way, by determining the target prediction weight through the number of symmetrical feature points, the first feature point, historical feature points, the first predicted feature point, and the second predicted feature point can be weighted and fused together with the 3D depth image to construct the 3D model of the target head, ensuring the accuracy and real-time performance of the 3D model.

[0016] In some implementations, the preset constraints include: The line connecting the first and second tragus points is parallel to the coronal plane of the head; and / or The distance from the tip of the nose to the tragus point on the first side is the same as the distance from the tip of the nose to the tragus point on the second side; and / or The vertical distance from the top of the head to the line connecting the first tragus point on the first side and the second tragus point on the second side is proportional to the height of the head in the sagittal plane.

[0017] Thus, the preset constraints include: the line connecting the first and second tragus points is parallel to the coronal plane of the head; the distance from the tip of the nose to the first tragus point is the same as the distance from the tip of the nose to the second tragus point; and the perpendicular distance from the top of the head to the line connecting the first and second tragus points is proportional to the height of the head in the sagittal plane. By clearly defining these preset constraints based on human physiological structure, a clear physiological structural standard can be provided for constructing the 3D model of the target head, ensuring the realism and accuracy of the 3D model. This provides a reliable foundation for subsequent head posture recognition, making the head posture recognition results more consistent with reality, and providing stable and accurate technical support for in-vehicle functions.

[0018] This application provides a vehicle including a memory and a processor. The memory stores a computer program, which, when executed by the processor, implements the steps of the above-described method.

[0019] This application provides a computer-readable storage medium storing a computer program that, when executed by one or more processors, implements the steps of the above-described method.

[0020] The vehicle and computer-readable storage medium provided in this application acquire a three-dimensional depth image of a target occupant's head; perform feature extraction processing on the three-dimensional depth image to determine head feature data; when any feature point in the head feature data is detected to be temporarily occluded, construct a three-dimensional model of the target occupant's head based on historical feature points corresponding to the missing feature points, the head feature data, and the three-dimensional depth image; and identify the target occupant's head posture based on the target head three-dimensional model. Thus, by acquiring multiple consecutive frames of target occupant head three-dimensional depth images, feature extraction processing can be performed on the three-dimensional depth images to obtain head feature data, and feature points in the head feature data can be detected. Even under temporary occlusion, historical feature points can be invoked to combine the head feature data and the three-dimensional depth image, accurately reconstructing the head contour details of the target occupant, constructing a complete and accurate target head three-dimensional model, and identifying the target occupant's head posture based on this target head three-dimensional model, improving the accuracy and anti-occlusion capability of head posture recognition, and providing accurate decision-making basis for subsequent vehicle functions.

[0021] Additional aspects and advantages of embodiments of this application will be set forth in part in the description which follows, and in part will be obvious from the description, or may be learned by practice of embodiments of this application. Attached Figure Description

[0022] The above and / or additional aspects and advantages of this application will become apparent and readily understood from the description of the embodiments taken in conjunction with the following drawings, wherein: Figure 1 This is one of the flowcharts illustrating a method for recognizing the head posture of an in-vehicle occupant according to certain embodiments of this application; Figure 2 This is a schematic diagram of the structure of a vehicle occupant head posture recognition device according to certain embodiments of this application; Figure 3 This is a schematic diagram showing the location of a three-dimensional imaging device according to certain embodiments of this application; Figure 4 This is a second schematic flowchart of a method for recognizing the head posture of occupants inside a vehicle according to certain embodiments of this application. Figure 5 This is the third flowchart of a method for recognizing the head posture of occupants inside a vehicle according to certain embodiments of this application; Figure 6 This is the fourth flowchart of a method for recognizing the head posture of occupants in a vehicle according to certain embodiments of this application. Figure 7 This is a schematic diagram of the head posture recognition process for vehicle occupants according to certain embodiments of this application. Detailed Implementation

[0023] The embodiments of this application are described in detail below. Examples of the embodiments are shown in the accompanying drawings, wherein the same or similar reference numerals denote the same or similar elements or elements having the same or similar functions throughout. The embodiments described below with reference to the accompanying drawings are exemplary and are only used to explain the embodiments of this application, and should not be construed as limiting the embodiments of this application.

[0024] By recognizing the head posture of vehicle occupants, data can be provided to support safe driving and intelligent cockpit interaction functions. For example, in the field of safety, obtaining occupant head posture data can assist intelligent airbag systems in adaptively adjusting their deployment strategy based on the occupant's head position and posture, avoiding secondary injuries. It can also accurately determine whether the driver is in a dangerous state such as looking down at a mobile phone or turning their head away in distraction, and issue timely warnings. In the field of cockpit entertainment and comfort, the sound field direction of the audio system, the angle of the display screen, and the cabin noise reduction parameters can be automatically adjusted based on head posture to provide occupants with a personalized experience.

[0025] However, the complex environment inside a vehicle cabin can easily cause occupants' heads to be obstructed, affecting recognition accuracy. On the one hand, inherent components such as the steering wheel, front sun visor, and A-pillars within the cabin create fixed obstruction areas; on the other hand, occupants' daily actions and habits can also cause obstruction problems, such as resting their chin on their hand while driving, holding a mobile phone or a drink, or wearing masks or glasses, which can obstruct key head features and make it impossible to accurately identify the occupant's head posture.

[0026] Based on the above issues, please refer to Figure 1This application provides a method for recognizing the head posture of vehicle occupants, the method comprising: 01: Acquire a three-dimensional depth image of the target occupant's head, wherein the three-dimensional depth image is acquired by the vehicle's three-dimensional imaging equipment; 02: Perform feature extraction processing on the 3D depth image to determine head feature data; 03: If any feature point in the head feature data is detected to be temporarily occluded, construct a three-dimensional model of the target occupant's head based on the historical feature points corresponding to the missing feature points, the head feature data, and the three-dimensional depth image. 04: Identify the target occupant's head posture based on the target head 3D model.

[0027] Please see Figure 2 This application provides a vehicle occupant head posture recognition device 100. The vehicle occupant head posture recognition method of this application can be implemented by the vehicle occupant head posture recognition device 100 of this application. Specifically, the vehicle occupant head posture recognition device 100 includes a three-dimensional imaging module 110, a feature extraction module 120, a three-dimensional model construction module 130, and a posture classification module 140. The three-dimensional imaging module 110 is used to acquire a three-dimensional depth image of the target occupant's head. The feature extraction module 120 is used to perform feature extraction processing on the three-dimensional depth image to determine head feature data, wherein the three-dimensional depth image is acquired by the vehicle's three-dimensional imaging device. The three-dimensional model construction module 130 is used to construct a target head three-dimensional model of the target occupant based on historical feature points corresponding to the missing feature points, head feature data, and the three-dimensional depth image when any feature point in the head feature data is detected to be temporarily occluded. The posture classification module 140 is used to identify the target head posture of the target occupant based on the target head three-dimensional model.

[0028] This application also provides a vehicle, which includes a memory and a processor. The vehicle occupant head pose recognition method of this application can be implemented by the vehicle of this application. Specifically, the memory stores a computer program, and the processor is used to acquire a three-dimensional depth image of the target occupant's head. The processor is also used to perform feature extraction processing on the three-dimensional depth image to determine head feature data, wherein the three-dimensional depth image is acquired by the vehicle's three-dimensional imaging device. The processor is also used to construct a three-dimensional model of the target occupant's head based on historical feature points corresponding to the missing feature points, the head feature data, and the three-dimensional depth image when any feature point in the head feature data is detected to be temporarily occluded. The processor is also used to identify the target head pose of the target occupant based on the three-dimensional head model.

[0029] Specifically, the target occupants refer to the people in the vehicle cabin who require head posture recognition, such as the driver, front passenger, and other types of people in the vehicle.

[0030] Three-dimensional depth images refer to structured image data containing three-dimensional spatial position information of the target occupant's head. Each pixel contains three-dimensional coordinates of horizontal, vertical and distance from the acquisition device, which are used to characterize the three-dimensional structure, contour and spatial distance relationship of the target occupant's head. They can be acquired by three-dimensional imaging devices such as Time of Flight (ToF) cameras. Three-dimensional depth images also contain multiple consecutive three-dimensional depth image frames to capture the dynamic changes of the target occupant's head.

[0031] Head feature data is a collection of feature points obtained by feature extraction from 3D depth images. Feature points are key points that can characterize head structure and posture, such as the tip of the nose, the alar of the nose, the visible corner of the eye, the tragus, the angle of the jaw, and the submental point. Each frame of head feature data corresponds to one frame of 3D depth image and contains zero or several feature points.

[0032] Temporary occlusion refers to a state in which the feature points of the target occupant's head cannot be detected by 3D depth images temporarily due to factors such as the occupant's own actions, posture changes, or objects being held, and a state in which the feature points could be detected by 3D depth images before the occlusion state.

[0033] Feature points that are temporarily occluded are considered missing feature points. For example, feature points such as the tip of the nose may disappear and cannot be extracted due to changes in the target occupant's posture, such as turning their head; feature points such as the jaw angle and subchinus may be occluded and cannot be extracted due to the target occupant wearing a mask or resting their chin on their hand; or the tragus may be occluded due to making or receiving a phone call.

[0034] Historical feature points are valid feature point data corresponding to the current missing feature point, which are stored in the header feature data before the missing feature point is temporarily occluded or in the historically stored header feature data.

[0035] The target head 3D model is a digital model reconstructed based on currently acquired head feature data, 3D depth images, and historical feature points. It can completely and accurately guarantee the three-dimensional shape, structure, proportion, and spatial position of the target occupant's head, and is used to identify the target head posture.

[0036] The target head posture refers to the motion state of the target occupant in space, including posture data such as pitch angle corresponding to head-down and head-up movements, yaw angle corresponding to head-turning movements, and roll angle corresponding to head-tilting movements.

[0037] By using vehicle-mounted 3D imaging equipment, such as a ToF camera, multiple consecutive 3D depth images of the target occupant's head can be acquired, providing basic data support for subsequent processing and avoiding recognition errors caused by the randomness of a single 3D depth image.

[0038] By performing feature extraction processing on each acquired 3D depth image frame, environmental interference can be eliminated, and the focus can be placed on the anatomical feature points in each 3D depth image frame that can represent the head structure information, thus determining the head feature data of multiple frames and laying the foundation for the subsequent construction of the target head 3D model.

[0039] The following explanations use a ToF camera as an example. In one example, the ToF camera can be placed on the side of the target occupant, such as on the A-pillar. The optical axis of the ToF camera lens can be at a 35° angle to the axis directly in front of the occupant. The ToF camera only needs to acquire a 3D depth image of the target occupant's side face and perform feature extraction processing. Based on algorithms such as curvature analysis and template matching, it can capture feature points on one side of the face, such as the side profile, a single ear, part of the forehead, and the bridge of the nose, to make symmetrical predictions based on the occupant's head symmetry, thereby supporting the establishment of a complete 3D model of the target head.

[0040] After acquiring head feature data, it is necessary to detect whether any feature points in the head feature data are in a temporary occlusion state. If any feature in the head feature data is detected to be in a temporary occlusion state, it can be considered that the current feature point cannot participate in the construction of the target head 3D model. For example, if the target occupant is making a hand-resting-chin gesture, causing the jaw angle to be occluded. By acquiring historical feature points corresponding to the missing feature points, the missing feature points can be filled in. This allows for the construction of the target occupant's 3D head model by combining the head feature data and 3D depth image, ensuring the completeness and accuracy of the target head 3D model. Furthermore, based on the target head 3D model, the target occupant's target head posture can be accurately identified, providing a decision-making basis for in-vehicle functions.

[0041] Understandably, when a feature point is detected in the head feature data that is not occluded, i.e. the feature point continues to exist, the feature point can be predicted directly based on the symmetry of the occupant's head, and the predicted feature point symmetrical to the feature point can be obtained and stored. When a feature point is detected in the head feature data that is temporarily occluded, the previously stored unoccluded historical feature points can be called to provide data basis for feature completion.

[0042] In one example, the vehicle occupant head posture recognition device 100 also includes a preprocessing module 150. After the three-dimensional imaging module 110 acquires a three-dimensional depth image of one side of the target occupant's face, the preprocessing module 150 is connected to the three-dimensional imaging module 110. It can perform noise reduction, distortion correction, and other processing on the three-dimensional depth image to eliminate interference from in-vehicle clutter and correct the error of the ToF camera, so as to avoid noise or distortion affecting the accuracy of subsequent feature extraction and model reconstruction. The preprocessing module 150 can also perform point cloud conversion processing on the three-dimensional depth image to obtain point cloud images, providing accurate data support for subsequent feature extraction processing to obtain head feature data.

[0043] The feature extraction module 120 also includes a symmetry feature inference module. The feature extraction module 120 is connected to the preprocessing module 150 and can perform feature extraction processing on the processed point cloud image, such as extracting head feature data, i.e. feature points, through curvature analysis, template matching or machine learning models. The symmetry feature inference module can perform symmetrical prediction on feature points in an unoccluded state to obtain the predicted feature points of the other side of the target occupant's face.

[0044] The 3D model construction module 130 includes a position calculation module 160 and a dynamic compensation module 170. The dynamic compensation module 170 can detect whether there are feature points that are temporarily occluded. Specifically, it can detect whether there are feature points in the head feature data that are in a temporary occlusion state. If any feature point in the head feature data is detected to be in a temporary occlusion state, it can call the historical feature points corresponding to the occluded missing feature points to make predictions, obtain the predicted feature points that are symmetrical to the historical feature points, and combine the feature points in the unoccluded state and their predicted feature points to jointly construct a complete target head 3D model.

[0045] The position calculation module 160 can use point cloud reconstruction algorithms to construct a complete three-dimensional model of the target head, and perform calculations based on the complete three-dimensional model of the target head, such as calculating the three-dimensional coordinates of the head center point with the center point of the vehicle's front axle as the origin, as well as attitude parameters such as pitch angle, yaw angle, and roll angle, so that the attitude classification module 140 connected to the position calculation module 160 can identify the head movements of the target occupant based on the three-dimensional coordinates of the head center point and attitude parameters such as pitch angle, yaw angle, and roll angle.

[0046] Furthermore, based on the rigid body transformation principle of head motion, the head rotation center can be accurately calculated by observing the spatial changes of head feature points, and the overall head shape can be inferred by combining the anatomical dimensions of the head, thereby constructing an accurate 3D model of the target head. First, after obtaining historical feature points and their symmetrical predicted feature points, unoccluded feature points, and their corresponding predicted feature points, when there are more than three pairs of non-collinear 3D feature points, a centering process is performed on all feature point sets in conjunction with a 3D depth image. For example, the centroid C1 of the initial feature point set and the centroid C2 of the post-motion feature point set are calculated separately, and then the coordinates of the corresponding centroid are subtracted from the coordinates of each point in both sets to obtain two sets of centered feature point sets, thus eliminating the interference of translation on the rotation matrix and translation vector. Then, the centered feature point sets are processed by... The optimal rotation matrix R is solved by constructing the covariance matrix and singular value decomposition, so as to accurately describe the head motion based on the rotation matrix R, i.e., calculate attitude parameters such as yaw angle, pitch angle, and roll angle. Then, based on the transformation relationship of the centroid of the centered feature point set, the head translation vector t can be directly calculated. Then, combined with the rotation matrix R and the translation vector t, all parameters of the rigid body transformation of the head can be calculated. Finally, combined with the standard anatomical dimensions of the human head, such as head width = distance between the left and right tragus points, and head height = distance from the top of the head to the submental point, the overall outline of the head can be reconstructed, such as the position of the ears.

[0047] In one example, if the pitch angle is greater than 30° and lasts for 0.5 seconds, the target occupant can be considered to be head-down; if the yaw angle is greater than 45° and lasts for 0.5 seconds, the target occupant can be considered to be head-turning; if the roll angle is greater than 15° and lasts for 0.5 seconds, the target occupant can be considered to be head-tilting; and if the change in the vertical axis (Z-axis) coordinate in three-dimensional coordinates is greater than 5 cm / s and lasts for 0.5 seconds, the target occupant can be considered to be leaning forward.

[0048] By recognizing the head movements of the target occupant based on a 3D model of the target head, the position and angle of vehicle components can be adjusted according to the occupant's head movements. For example, intelligent airbag systems can adaptively adjust the deployment strategy based on the occupant's position and posture to avoid secondary injuries; driver status monitoring can determine whether the driver is distracted or fatigued by head posture; and personalized entertainment and comfort systems can automatically adjust the sound field of the audio system or the angle of the display screen based on the occupant's head position to enhance the in-vehicle occupant experience.

[0049] Compared to situations where the accuracy of posture recognition is low due to temporary obstruction of the target occupant, the implementation method of this application can call historical feature points in the state of temporary obstruction, and combine head feature data and three-dimensional depth images to accurately restore the head contour details of the target occupant, construct a complete and accurate three-dimensional model of the target head, and finally recognize the head posture of the target occupant based on the three-dimensional model of the target head, thereby improving the accuracy of head posture recognition and providing accurate decision-making basis for subsequent vehicle functions.

[0050] In summary, this application's implementation method acquires multiple consecutive frames of three-dimensional depth images of the target occupant's head. Feature extraction processing is performed on these three-dimensional depth images to obtain head feature data, and feature points within this data are detected. Even when feature points are temporarily occluded, a complete three-dimensional model of the target head can be constructed based on historical feature points combined with current head feature data and the three-dimensional depth image. This allows for occupant head posture recognition based on the model representing the target head, improving the accuracy of head posture recognition. Compared to situations where temporary occupant ...

[0051] In some embodiments, at least one three-dimensional imaging device is installed in the vehicle cabin; and / or The 3D imaging device is positioned at the front left of the vehicle's cockpit; and / or The 3D imaging equipment is installed in the front left ceiling of the vehicle's cabin; and / or The 3D imaging device is positioned on the left front A-pillar of the vehicle's cabin; and / or The 3D imaging device is installed on the left B-pillar of the vehicle's cabin; and / or The 3D imaging device is positioned at the front right of the vehicle's cockpit; and / or The 3D imaging equipment is installed in the right front ceiling of the vehicle cabin; and / or The 3D imaging device is positioned on the right front A-pillar of the vehicle's cabin; and / or The 3D imaging device is installed on the right-side B-pillar of the vehicle's cabin; and / or The field of view of the 3D imaging device is a preset field of view, which is 80°-100°; and / or The optical axis of the lens of the 3D imaging device is at a preset angle to the axis directly in front of the vehicle cabin, with the preset angle being 30°-70°; and / or The center point of the lens of the 3D imaging device is at a preset vertical distance from the seat cushion plane of the vehicle cabin, the preset vertical distance being 0.5 meters to 0.7 meters; and / or The center point of the lens of the 3D imaging device is at a preset horizontal distance from the center line of the shoulder of the vehicle cabin, which is 0.4 meters to 0.6 meters.

[0052] Specifically, the vehicle cabin is equipped with at least one three-dimensional imaging device, namely a three-dimensional imaging module 110, i.e. Figure 3 The ToF camera shown The 3D imaging device can be set at the front left of the vehicle cabin to acquire a 3D depth image of the left side of the occupant's head, such as the front left roof, the front left A-pillar, and the left B-pillar; the 3D imaging device can be set at the front right of the vehicle cabin to acquire a 3D depth image of the right side of the occupant's head, such as the front right roof, the front right A-pillar, and the right B-pillar, so as to cover the occupant's head area from a side and above perspective, avoiding obstructions such as the steering wheel, hand movements, and sun visors, while staying away from the direct path of strong frontal light and reducing ambient light interference.

[0053] The 3D imaging device can also be placed near the front center rearview mirror in the vehicle cabin, away from the front of the occupants. There are no restrictions on this, and the position of the 3D imaging device can be set according to the actual usage.

[0054] The field of view of the 3D imaging device can be set to a preset field of view of 80°-100°. The preset field of view can be set according to the actual use situation, such as 80°, 85°, 90° or 100°, to cover the complete three-dimensional area of ​​the occupant's head from the shoulders to the top of the head. This will prevent the head edge features from being missed due to the angle being too small, and will also prevent irrelevant data such as the cabin interior and seats from being introduced due to the angle being too large.

[0055] The angle between the lens optical axis of the 3D imaging device and the front axis of the vehicle cabin can be set to a preset angle of 30°-70°. The preset angle can be set according to the actual use, such as 30°, 35°, 47°, 50.3°, and 70°, so that the device can capture the side and some front features of the head from the best side and top perspective, ensuring that the three-dimensional structure is clearly presented.

[0056] The distance between the center point of the 3D imaging device's lens and the seat cushion plane of the vehicle cabin can be set to a preset vertical distance of 0.5 meters to 0.7 meters, which can be adjusted according to actual usage, such as 0.5 meters, 0.6 meters, 0.64 meters, or 0.7 meters. Similarly, the distance between the center point of the 3D imaging device's lens and the shoulder centerline of the vehicle cabin can be set to a preset horizontal distance of 0.4 meters to 0.6 meters, which can also be adjusted according to actual usage, such as 0.4 meters, 0.56 meters, 0.56 meters, or 0.6 meters. This avoids image distortion due to excessively close distances or blurring of feature points due to excessively large distances, ensuring the accuracy of the acquired 3D depth image.

[0057] By setting the configuration, location, and parameters of the 3D imaging equipment, interference factors from frontal placement can be avoided, ensuring that accurate and reliable 3D depth images can be acquired in complex in-vehicle environments, laying a reliable foundation for subsequent feature processing and the establishment of a 3D model of the target head.

[0058] Furthermore, compared to the instability of visual cameras in strong or low light environments, ToF cameras are unaffected by factors such as lighting and can provide accurate and reliable 3D depth images for subsequent feature extraction in complex environments.

[0059] It should be noted that the location and angle of the 3D imaging device in the embodiments of this application are only illustrative and should not be construed as limiting the values. In other examples, the settings can be adjusted according to the actual situation.

[0060] Thus, at least one 3D imaging device is installed in the vehicle cabin; the 3D imaging device is located at the left front of the vehicle cabin; the 3D imaging device is located at the left front of the vehicle cabin roof; the 3D imaging device is located at the left front of the vehicle cabin A-pillar; the 3D imaging device is located at the left side of the vehicle cabin B-pillar; the 3D imaging device is located at the right front of the vehicle cabin; the 3D imaging device is located at the right front of the vehicle cabin roof; the 3D imaging device is located at the right front of the vehicle cabin A-pillar; the 3D imaging device is located at the right side of the vehicle cabin B-pillar; the field of view of the 3D imaging device is a preset field of view, which is 80°-100°; the optical axis of the lens of the 3D imaging device is at a preset angle to the front axis of the vehicle cabin, which is 30°-70°; the center point of the lens of the 3D imaging device is at a preset vertical distance from the seat cushion plane of the vehicle cabin, which is 0.5 meters-0.7 meters; the center point of the lens of the 3D imaging device is at a preset horizontal distance from the shoulder centerline of the vehicle cabin, which is 0.4 meters-0.6 meters. In this way, by setting the configuration, position and parameters of the 3D imaging equipment, interference factors of frontal placement can be avoided, ensuring that accurate and reliable 3D depth images can be acquired in complex in-vehicle environments, laying a reliable foundation for subsequent feature processing and the establishment of a 3D model of the target head.

[0061] In some implementations, the head feature data includes reference feature points and symmetrical feature points; the reference feature points include the tip of the nose, the root of the nose, and the apex of the chin; the symmetrical feature points include the tragus, the auricle, the vertex of the head, the alar of the nose, the angle of the mandible, the submental point, the canthus of the eye, the orbital point, and / or the occipital protuberance.

[0062] Specifically, the head feature data includes reference feature points and symmetrical feature points. Reference feature points refer to feature points that can construct the symmetrical reference plane of the target occupant's head, including feature points located on the symmetrical reference plane of the head such as the tip of the nose, the root of the nose, and the chin. Symmetrical feature points refer to feature points other than the baseline feature points, including symmetrical feature points such as the tragus point, auricle point, vertex of the head, nasal ala point, angle of the mandible point, submental point, canthus point, orbital point, and / or occipital protuberance point.

[0063] By clearly defining the reference feature points, including the tip of the nose, the root of the nose, and the chin, we can lay the foundation for the subsequent construction of a symmetrical reference plane.

[0064] Understandably, a reference feature point refers to a feature point located on the symmetrical reference plane of the head. In other examples, the reference feature point can also be the top of the head, the occipital protuberance, etc., which can be set according to the actual situation.

[0065] Symmetrical feature points include the tragus point, auricle point, vertex of the head, ala nasi point, angle of the mandible point, submental point, canthus point, orbital point, and / or occipital protuberance point, covering the sides, top, face, lower part, and back of the head. The tragus and auricle points characterize the upper structures of the head's sides; the ala nasi point, canthus point, and orbital point refine the three-dimensional facial contours; the angle of the mandible and submental point support the reconstruction of the lower head structures; and the vertex of the head and occipital protuberance point supplement the longitudinal height and posterior structural references of the head, respectively. Based on comprehensive and accurate symmetric feature points, it is ensured that each region of the head has a corresponding symmetric mapping basis, providing accurate data for subsequent construction of the target head 3D model.

[0066] Thus, the head feature data includes baseline feature points and symmetrical feature points. Baseline feature points include the tip of the nose, the root of the nose, and the chin. Symmetrical feature points include the tragus, the auricle, the vertex of the head, the ala of the nose, the angle of the mandible, the submental point, the canthus of the eye, the orbital point, and / or the occipital protuberance. By clearly defining the baseline feature points, a foundation can be laid for constructing a symmetrical reference plane. Furthermore, by clearly defining the symmetrical feature points, it can be ensured that each region of the head has a corresponding symmetrical mapping basis, providing accurate data for subsequently building a 3D model of the target head.

[0067] Please see Figure 4 In some implementations, the symmetrical feature points include the missing feature points on the first side and the first feature points on the first side excluding the missing feature points. Step 03 (when any feature point in the head feature data is detected to be in a temporary occlusion state, constructing a three-dimensional model of the target occupant's head based on the historical feature points corresponding to the missing feature points in the temporary occlusion state, the head feature data, and the three-dimensional depth image) includes: 031: Determine the symmetrical reference plane based on the reference feature points; 032: When missing feature points are detected to be missing continuously within a preset time, based on the symmetric reference plane, the historical feature points are symmetrically mapped to determine the first predicted feature point in the second side that is symmetrical to the historical feature points, wherein the second side is symmetrical to the first side. 033: Based on preset constraints, construct a three-dimensional model of the target head according to the first feature point, historical feature points, first predicted feature points and three-dimensional depth image.

[0068] In some embodiments, the symmetry feature prediction module 121 is used to determine a symmetry reference plane based on reference feature points. The symmetry feature prediction module 121 is also used to, when missing feature points are detected to be continuously missing within a preset time, perform symmetry mapping processing on historical feature points based on the symmetry reference plane to determine a first predicted feature point on the second side that is symmetrical to the historical feature points, wherein the second side is symmetrical to the first side. The position calculation module 160 is used to construct a three-dimensional model of the target head based on preset constraints, the first feature point, the historical feature points, the first predicted feature point, and the three-dimensional depth image.

[0069] In some implementations, the processor is further configured to determine a symmetry reference plane based on reference feature points. The processor is also configured to, upon detecting consecutive missing feature points within a preset time period, perform symmetry mapping processing on historical feature points based on the symmetry reference plane to determine a first predicted feature point symmetrical to the historical feature points on a second side, wherein the second side is symmetrical to the first side. The processor is further configured to construct a three-dimensional model of the target head based on preset constraints, using the first feature point, historical feature points, the first predicted feature point, and a three-dimensional depth image.

[0070] Specifically, the first side refers to the side of the head that can be directly detected by a 3D imaging device when the head is symmetrical. For example, if the ToF camera is placed on the driver's left A-pillar, the driver's left side of the face is the first side.

[0071] The first feature point is the feature point that can be directly detected by a three-dimensional imaging device, that is, the first feature point on the first side excluding the missing feature point.

[0072] The second side is the other side of the head that is symmetrical to the first side. The feature points of the second side cannot be extracted directly and need to be inferred through symmetry mapping.

[0073] The symmetry reference plane is a plane constructed based on reference feature points. It is used to accurately divide the head into two symmetrical parts, providing a reference for subsequent symmetrical mapping of historical feature points to obtain the first predicted feature point.

[0074] The preset time is a pre-set time threshold, such as 10 frames, used to determine whether feature points are continuously missing. It can be set according to the actual situation.

[0075] The first predicted feature point is obtained by symmetrically mapping the historical feature points of the first side along the symmetry reference plane, and the corresponding feature points of the second side are used to complete the missing feature information of the second side according to the symmetry of the head.

[0076] The preset constraints are rules set based on the anatomical structure of the human head. They are used to regulate the location of predicted feature points and the construction logic of the target head 3D model to ensure that the target head 3D model conforms to the actual structural proportions of the human head.

[0077] Based on the benchmark feature points located on the central axis of the front of the head, such as the tip of the nose, the root of the nose, and the chin, a symmetrical reference plane can be constructed through spatial geometric algorithms. This provides an accurate and stable reference for the symmetrical mapping of subsequent feature points, ensuring that the predicted feature points after mapping conform to the actual structural proportions of the head.

[0078] By setting a condition that a feature point is in a temporary occlusion state based on continuous absence within a preset time, it is determined that a symmetrical feature point on the first side is temporarily occluded only when it cannot be detected continuously within the preset time. The feature point is then identified as a missing feature point. Based on the symmetrical reference plane, the historical feature points corresponding to the missing feature points are symmetrically mapped to determine the first predicted feature point on the second side that is symmetrical to the historical feature points. For example, the preset time can be set as a time window of 4 consecutive frames. If a feature point does not appear within 4 consecutive frames, that is, when the number of visible feature points decreases by ≥3 within 4 consecutive frames, it can be determined that there is a feature point in a temporary occlusion state.

[0079] Understandably, the head has a left-right symmetrical structure. The spatial positions of the missing feature points on the first side and the corresponding feature points on the second side are symmetrical about the symmetry reference plane. By mirror transformation, that is, by symmetrical mapping of the historical feature points corresponding to the missing feature points, the first predicted feature point in the second side that is symmetrical to the historical feature point can be accurately generated. The historical feature point can replace the missing feature point on the first side that is occluded, and the first predicted feature point can replace the symmetrical feature point of the missing feature point on the second side, thus completing the feature information.

[0080] Finally, by integrating the first feature point, historical feature points, first predicted feature points, three-dimensional depth image data, and preset constraints, an accurate and continuous three-dimensional model of the target head is constructed, avoiding interruption of target occupant head posture recognition due to temporary occlusion.

[0081] In one example, the symmetric feature inference module can use the Random Sample Consensus (RANSAC) algorithm to remove outlier feature points in the head feature data, providing accurate data for constructing the symmetric reference surface and symmetric mapping.

[0082] The dynamic compensation module 170 includes an occlusion detection unit 171 and a historical pose matching unit 172. The occlusion detection module can be used to identify whether a feature point is in a temporary occlusion state. For example, by setting a preset time, the changes in the feature points in consecutive frames within the preset time can be used to determine whether a temporary occlusion exists. The historical pose matching unit 172 can call the historical feature points corresponding to the missing feature points when it detects that missing feature points are continuously missing within a preset time. Based on dynamic data fusion and optimization algorithms such as Kalman wave, it combines the first feature point and the three-dimensional depth image to build a three-dimensional model of the target head, ensuring the continuity and smoothness of the three-dimensional model of the target head.

[0083] Thus, a symmetrical reference plane is determined based on the baseline feature points. When missing feature points are detected to be continuously missing within a preset time period, symmetrical mapping is performed on historical feature points based on the symmetrical reference plane to determine the first predicted feature point symmetrical to the historical feature points on the second side, where the second side is symmetrical to the first side. Based on preset constraints, a 3D model of the target head is constructed using the first feature point, historical feature points, the first predicted feature point, and the 3D depth image. In this way, by constructing a symmetrical reference plane using baseline feature points, symmetrical mapping can be performed on historical feature points corresponding to continuously missing feature points within a preset time period to generate the first predicted feature point. Then, combining the first feature point, historical feature points, the 3D depth image, and preset constraints, a complete 3D model of the target head is constructed, solving the problems of inaccurate feature point completion and model reconstruction distortion under occlusion conditions, and improving the accuracy of head pose recognition based on the 3D model of the target head.

[0084] Please see Figure 5 In some implementations, step 033 (constructing a 3D model of the target head based on preset constraints, using the first feature point, historical feature points, first predicted feature points, and a 3D depth image) includes: 0331: Based on the symmetric reference plane, the first feature point is symmetrically mapped to determine the second predicted feature point on the second side that is symmetric to the first feature point; 0332: Based on preset constraints, construct a three-dimensional model of the target head according to the first feature point, historical feature points, first predicted feature points, second predicted feature points and three-dimensional depth image.

[0085] In some embodiments, the symmetry feature prediction module 121 is further configured to perform symmetry mapping processing on the first feature point based on a symmetry reference plane to determine a second predicted feature point on the second side that is symmetrical to the first feature point. The position calculation module 160 is further configured to construct a three-dimensional model of the target head based on preset constraints, the first feature point, historical feature points, the first predicted feature point, the second predicted feature point, and the three-dimensional depth image.

[0086] In some implementations, the processor is further configured to perform symmetric mapping processing on the first feature point based on a symmetric reference plane to determine a second predicted feature point on the second side that is symmetrical to the first feature point. The processor is also configured to construct a three-dimensional model of the target head based on preset constraints, using the first feature point, historical feature points, the first predicted feature point, the second predicted feature point, and a three-dimensional depth image.

[0087] Specifically, the second predicted feature point is a predictive feature point obtained on the second side, which is symmetrical to the first side, after performing symmetrical mapping on the first feature point that is not occluded on the first side, based on the symmetry reference plane. The second predicted feature point is symmetrical to the first feature point about the symmetry reference plane and is used to complete the feature information of the corresponding position on the second side, ensuring the integrity of the target head 3D model.

[0088] Based on the first feature point that is not occluded on the first side, a symmetrical mapping process can be performed using a spatial geometric mirroring algorithm to obtain a second predicted feature point that is precisely symmetrical to the first feature point. By combining the first feature point, historical feature points, the first predicted feature point, and the three-dimensional depth image, a three-dimensional model of the target head can be constructed, ensuring the integrity of the three-dimensional model of the target head and improving the accuracy of head posture recognition based on the three-dimensional model of the target head.

[0089] In one example, in the absence of missing feature points that are temporarily occluded, the first feature point can be directly mapped to the second side based on the symmetry reference plane to obtain the second predicted feature point. Based on preset constraints, a three-dimensional model of the target head is constructed according to the first feature point, the second predicted feature point, and the three-dimensional depth image. Then, the head posture of the target occupant is recognized based on the three-dimensional model of the target head.

[0090] In one example, the 3D depth image acquired by the ToF camera may also include images from the second side, meaning that the head feature data also contains second-side feature points. After symmetrically mapping the historical feature points and the first feature points from the first side, the detected second-side feature points can be used to verify the first and second predicted feature points obtained from the symmetrical mapping, thereby optimizing the target head 3D model and improving the accuracy of the target head 3D model to a certain extent.

[0091] In one example, when accuracy requirements are not high, the pre-fitted 3D head model can be adjusted or configured using the first feature point, historical feature points, first predicted feature points, and second predicted feature points to improve computation speed and adapt to low-precision scenarios where real-time performance is prioritized.

[0092] By performing symmetrical mapping on the first feature point that is not occluded on the first side, the second predicted feature point can be determined to complete the feature information of the second side. Then, the first feature point, historical feature point, first predicted feature point, second predicted feature point and three-dimensional depth image are fused to construct a three-dimensional model of the target head under preset constraints, ensuring the integrity and accuracy of the three-dimensional model of the target head, thereby improving the accuracy of head posture recognition based on the three-dimensional model of the target head to a certain extent.

[0093] Thus, based on a symmetrical reference plane, the first feature point is symmetrically mapped to determine a second predicted feature point on the second side that is symmetrical to the first feature point. Based on preset constraints, a 3D model of the target head is constructed using the first feature point, historical feature points, first predicted feature points, second predicted feature points, and a 3D depth image. In this way, by performing symmetrical mapping on the unoccluded first feature point on the first side, the second predicted feature point can be determined to complete the feature information of the second side. Furthermore, by fusing the first feature point, historical feature points, first predicted feature points, second predicted feature points, and the 3D depth image, a 3D model of the target head is constructed under preset constraints, ensuring the integrity and accuracy of the 3D model and thus improving the accuracy of head pose recognition based on the 3D model of the target head to a certain extent.

[0094] Please see Figure 6 In some implementations, the method further includes: 034: Determine the target prediction weight based on the number of symmetrical feature points; 035: If missing feature points are detected to be missing continuously within a preset time, a three-dimensional model of the target head is constructed based on preset constraints, target prediction weights, first feature points, historical feature points, first predicted feature points, second predicted feature points, and a three-dimensional depth image.

[0095] In some embodiments, the dynamic compensation module 170 further includes a symmetry weight adjustment unit 173, which is used to determine the target prediction weight based on the number of symmetric feature points. The position calculation module 160 is also used to construct a three-dimensional model of the target head based on preset constraints, the target prediction weight, the first feature point, historical feature points, the first predicted feature point, the second predicted feature point, and the three-dimensional depth image, when the missing feature points are detected to be missing continuously within a preset time.

[0096] In some implementations, the processor is further configured to determine the target prediction weight based on the number of symmetrical feature points. The processor is also configured to, upon detecting consecutive missing feature points within a preset time period, construct a three-dimensional model of the target head based on preset constraints, the target prediction weight, a first feature point, historical feature points, a first predicted feature point, a second predicted feature point, and a three-dimensional depth image.

[0097] Specifically, the target prediction weight is used to adjust the quantitative parameters of the influence of the first feature point, historical feature points, first predicted feature point, and second predicted feature point in the construction of the 3D model, and can be calculated from the number of currently acquired symmetrical feature points.

[0098] The magnitude of the target prediction weight is positively correlated with the number of symmetrical feature points. The more feature points there are, the higher the weight, which means the greater the credibility and reference value of the predicted feature points. Conversely, the fewer feature points there are, the lower the weight, and the influence of the predicted feature points needs to be reduced, relying more on measured data and historical stable data.

[0099] By counting the number of symmetrical feature points, the target prediction weight can be calculated based on a preset weight mapping rule.

[0100] In one example, when the total number of symmetrical feature points is large, the current feature information can be considered sufficient, and a higher target prediction weight can be set. For example, when the number of symmetrical feature points is greater than or equal to 5, the target prediction weight of the first feature point should be set to no less than 0.8. When the total number of symmetrical feature points is small, the current feature information can be considered scarce, and a lower target prediction weight can be set. That is, the target prediction weights of the first and second predicted feature points should be reduced, while the target prediction weights of the actual first feature point and the stable historical feature points should be increased to reduce the interference of low-confidence prediction data on the model.

[0101] In one example, if the number of first feature points is greater than or equal to 5, the feature information can be considered sufficient and the mapping accuracy of the second predicted feature point is high. The target prediction weights of the first feature point and the second predicted feature point can be assigned a higher value, such as 0.7-0.9. However, if the number of missing feature points is greater than or equal to 2, that is, if the number of first feature points is less than 4, the weights of the historical feature points corresponding to the missing feature points and the first predicted feature points can be reduced, and the target prediction weights of the first feature point and the second predicted feature point can be assigned a higher value.

[0102] Based on the target prediction weights, the first feature point, historical feature points, first predicted feature points, and second predicted feature points can be weighted and fused. For example, the calculation result of the first predicted feature point can be multiplied by its corresponding target prediction weight. Alternatively, the relative weights of the first and historical feature points can be adjusted inversely based on the target prediction weights; for instance, when the target prediction weight is 0.8, the combined relative weight of the first and historical feature points is 0.2. By combining the weighted fused feature data with the overall contour information provided by the 3D depth image, and then performing error calibration through preset constraints, deviations that may occur during the weighted fusion process can be corrected. Ultimately, a target head 3D model that conforms to the physiological structure of the head and is adapted to the current number of feature points is constructed, ensuring the accuracy and stability of the target head 3D model.

[0103] Thus, the target prediction weight is determined based on the number of symmetrical feature points. If missing feature points are detected to be continuously missing within a preset time period, a 3D model of the target head is constructed based on preset constraints, the target prediction weight, the first feature point, historical feature points, the first predicted feature point, the second predicted feature point, and the 3D depth image. In this way, by determining the target prediction weight through the number of symmetrical feature points, the first feature point, historical feature points, the first predicted feature point, and the second predicted feature point can be weighted and fused together with the 3D depth image to construct the 3D model of the target head, ensuring the accuracy and real-time performance of the 3D model.

[0104] In some implementations, the preset constraints include: The line connecting the first and second tragus points is parallel to the coronal plane of the head; and / or The distance from the tip of the nose to the tragus point on the first side is the same as the distance from the tip of the nose to the tragus point on the second side; and / or The vertical distance between the top of the head and the line connecting the first and second tragus points is proportional to the height of the head in the sagittal plane.

[0105] Specifically, the coronal plane of the head refers to a virtual plane that is perpendicular to the front-back direction of the human head and parallel to the left-right direction. It is the reference plane used in human anatomy to divide the front and back regions of the head.

[0106] The sagittal plane of the head is a virtual plane that is perpendicular to the left-right direction of the human head, parallel to the front-back direction, and passes through the central axis of the head. It is the reference plane for dividing the left and right symmetrical regions of the head.

[0107] The preset ratio is a fixed value pre-set based on human head anatomy statistics. It is used to constrain the proportional relationship of key longitudinal dimensions of the head and ensure that the model conforms to the physiological characteristics of the human head.

[0108] The line connecting the tragus points on the first and second sides can represent the lateral posture of the head. By constraining the line connecting the two tragus points to be parallel to the coronal plane, it can be ensured that the lateral structure of the head model conforms to the physiological characteristics of the human body, and avoid head tilt modeling caused by occlusion and mapping errors.

[0109] In one example, if the mapped position of the second tragus point is offset, it may cause the line to form an angle with the coronal plane. By automatically calibrating the three-dimensional coordinates of the second tragus point until the parallel constraint is met, the foundation for the overall symmetrical structure of the head can be laid.

[0110] The distance from the tip of the nose to the two tragus points is theoretically equal. By quantifying the distance relationship between the tip of the nose and the first and second tragus points, the symmetry accuracy of both sides of the face can be calibrated to ensure that the symmetry relationship on both sides of the facial midline conforms to the real human body structure and correct facial asymmetry deviations that may occur during symmetry mapping or weight fusion.

[0111] In one example, if the distance from the tip of the nose to the tragus point on one side exceeds the preset error range of 3mm on the other side, the coordinates of the tragus point on that side can be automatically adjusted to ensure that the symmetry between the two sides of the facial midline conforms to the real human body structure and correct facial asymmetry deviations that may occur during symmetry mapping or weight fusion.

[0112] By limiting the vertical distance from the top of the head to the line connecting the first and second tragus points to a preset ratio with the height of the head in the sagittal plane, the longitudinal contour of the head model can be ensured to conform to human proportions.

[0113] In one example, if the mapped top of the head is too high or too low, the vertical distance between it and the line connecting the two tragus will deviate from the preset ratio. By combining the head sagittal plane height calculated from the reference feature points such as the tip of the nose and the chin, the coordinates of the top of the head and related feature points can be calibrated in reverse to avoid distortion of head shape modeling and to adapt to occupants of different heights and head shapes.

[0114] Constraints verify and correct feature point coordinates and model structure from different dimensions, which can match the fused data with the head structure pattern, ensuring the authenticity and accuracy of the target head 3D model, providing a reliable foundation for subsequent head posture recognition, making the head posture recognition results more consistent with the actual situation, and providing stable and accurate technical support for vehicle functions.

[0115] Thus, the preset constraints include: the line connecting the first and second tragus points is parallel to the coronal plane of the head; the distance from the tip of the nose to the first tragus point is the same as the distance from the tip of the nose to the second tragus point; and the perpendicular distance from the top of the head to the line connecting the first and second tragus points is proportional to the height of the head in the sagittal plane. By clearly defining these preset constraints based on human physiological structure, a clear physiological structural standard can be provided for constructing the 3D model of the target head, ensuring the realism and accuracy of the 3D model. This provides a reliable foundation for subsequent head posture recognition, making the head posture recognition results more consistent with reality, and providing stable and accurate technical support for in-vehicle functions.

[0116] The following is Figure 7 Taking an example, the vehicle occupant head posture recognition process of the embodiments of this application will be explained: First, raw data, namely three-dimensional depth image data, is acquired through the three-dimensional imaging module 110; Then, the preprocessing module 150 performs noise reduction and distortion correction on the original data to obtain three-dimensional point cloud data; Next, the feature extraction module 120 can extract head feature points, i.e. head feature data, from the 3D point cloud data, establish a head reference plane, i.e. a symmetric reference plane, and evaluate its confidence level to determine the target prediction weight. Then, the position calculation module 160 can construct a 3D model of the head, i.e., a 3D model of the target head, based on the head feature data and the 3D point cloud data. Furthermore, the dynamic compensation module 170 can determine whether there is occlusion in the target head 3D model during the construction process, and process the data according to the occlusion determination result. If occlusion exists, the current data is optimized based on historical head pose data, i.e., historical feature points; if there is no occlusion, the weight of the currently detected feature points is increased. Finally, based on the target head 3D model, the posture classification module 140 judges the typical head movements of the current occupant and determines the in-vehicle position data of the current occupant's head according to historical posture data, so as to provide accurate decision-making basis for in-vehicle functions.

[0117] This application also provides a computer-readable storage medium having a computer program stored thereon. When the computer program processor executes, it implements the steps of the vehicle occupant head posture recognition method described above.

[0118] This application also provides a computer program product, including a computer program / instructions. When the computer program / instructions are executed by a processor, the steps of the vehicle occupant head posture recognition method described above can be implemented.

[0119] It is understood that a computer program includes computer program code. Computer program code can be in the form of source code, object code, executable files, or some intermediate form. Computer-readable storage media can include: any entity or device capable of carrying computer program code, recording media, USB flash drives, portable hard drives, magnetic disks, optical disks, computer memory, read-only memory (ROM), random access memory (RAM), and software distribution media, etc.

[0120] In this specification, the terms "specifically," "furthermore," "particularly," "understandably," etc., refer to specific features, structures, materials, or characteristics described in connection with embodiments or examples that are included in at least one embodiment or example of this application. In this specification, the illustrative expressions of the above terms do not necessarily refer to the same embodiment or example. Furthermore, the specific features, structures, materials, or characteristics described may be combined in any suitable manner in one or more embodiments or examples. Moreover, without contradiction, those skilled in the art can combine and integrate the different embodiments or examples described in this specification, as well as the features of different embodiments or examples.

[0121] Any process or method description in the flowchart or otherwise herein can be understood as representing a module, segment, or portion of executable request code comprising one or more steps for implementing a particular logical function or process, and the scope of the preferred embodiments of this application includes additional implementations in which functions may be performed not in the order shown or discussed, including substantially simultaneously or in reverse order according to the functions involved, as should be understood by those skilled in the art to which embodiments of this application pertain.

[0122] Although embodiments of this application have been shown and described above, it is understood that the above embodiments are exemplary and should not be construed as limiting this application. Those skilled in the art can make changes, modifications, substitutions and variations to the above embodiments within the scope of this application.

Claims

1. A method for recognizing the head posture of occupants inside a vehicle, characterized in that, The method includes: A three-dimensional depth image of the head of the target occupant is acquired, wherein the three-dimensional depth image is acquired by the three-dimensional imaging device of the vehicle; The three-dimensional depth image is subjected to feature extraction processing to determine head feature data; If any feature point in the head feature data is detected to be in a temporary occlusion state, a three-dimensional model of the target occupant's head is constructed based on the historical feature points corresponding to the missing feature points in the temporary occlusion state, the head feature data, and the three-dimensional depth image. Based on the three-dimensional model of the target head, the target occupant's target head posture is identified.

2. The method according to claim 1, characterized in that, The vehicle cabin is equipped with at least one three-dimensional imaging device; and / or The three-dimensional imaging device is located at the left front of the vehicle cabin; and / or The three-dimensional imaging device is installed on the left front ceiling of the vehicle cabin; and / or The three-dimensional imaging device is installed on the left front A-pillar of the vehicle cabin; and / or The three-dimensional imaging device is installed on the left B-pillar of the vehicle cabin; and / or The three-dimensional imaging device is located at the right front of the vehicle cabin; and / or The three-dimensional imaging device is installed on the right front ceiling of the vehicle cabin; and / or The three-dimensional imaging device is installed on the right front A-pillar of the vehicle cabin; and / or The three-dimensional imaging device is installed on the right-side B-pillar of the vehicle cabin; and / or The field of view of the three-dimensional imaging device is a preset field of view, which is 80°-100°; and / or The optical axis of the lens of the three-dimensional imaging device is at a preset angle to the axial direction directly in front of the vehicle cabin; the preset angle is 30°-70°; and / or The center point of the lens of the three-dimensional imaging device is at a preset vertical distance from the seat cushion plane of the vehicle cabin, the preset vertical distance being 0.5 meters to 0.7 meters; and / or The center point of the lens of the three-dimensional imaging device is at a preset horizontal distance from the center line of the shoulder of the vehicle cabin, and the preset horizontal distance is 0.4 meters to 0.6 meters.

3. The method according to claim 1, characterized in that, The head feature data includes baseline feature points and symmetrical feature points; the baseline feature points include the tip of the nose, the root of the nose, and the apex of the chin; the symmetrical feature points include the tragus, the auricle, the apex of the head, the ala of the nose, the angle of the mandible, the submental point, the canthus of the eye, the orbital point, and / or the occipital protuberance.

4. The method according to claim 3, characterized in that, The symmetrical feature points include the missing feature points on the first side and the first feature points on the first side excluding the missing feature points. When it is detected that any feature point in the head feature data is in a temporary occlusion state, a target head 3D model of the target occupant is constructed based on the historical feature points corresponding to the missing feature points in the temporary occlusion state, the head feature data, and the 3D depth image, including: Based on the aforementioned reference feature points, determine the symmetrical reference plane; If the missing feature points are detected to be missing continuously within a preset time, the historical feature points are symmetrically mapped based on the symmetrical reference plane to determine the first predicted feature point in the second side that is symmetrical to the historical feature points, wherein the second side is symmetrical to the first side. Based on preset constraints, a three-dimensional model of the target head is constructed according to the first feature point, the first predicted feature point, the historical feature point, and the three-dimensional depth image.

5. The method according to claim 4, characterized in that, The step of constructing a 3D model of the target head based on preset constraints, using the first feature point, the first predicted feature point, the historical feature point, and the 3D depth image, includes: Based on the symmetry reference plane, the first feature point is subjected to symmetry mapping processing to determine the second predicted feature point on the second side that is symmetric to the first feature point; Based on preset constraints, a three-dimensional model of the target head is constructed according to the first feature point, the first predicted feature point, the historical feature point, the second predicted feature point, and the three-dimensional depth image.

6. The method according to claim 5, characterized in that, The method further includes: The target prediction weight is determined based on the number of the symmetrical feature points; If the missing feature points are detected to be missing continuously within the preset time period, the target head three-dimensional model is constructed based on the preset constraints, the target prediction weight, the first feature point, the historical feature point, the first predicted feature point, the second predicted feature point, and the three-dimensional depth image.

7. The method according to any one of claims 4-6, characterized in that, The preset constraints include: The line connecting the first and second tragus points is parallel to the coronal plane of the head; and / or The distance from the tip of the nose to the tragus point on the first side is the same as the distance from the tip of the nose to the tragus point on the second side; and / or The vertical distance from the top of the head to the line connecting the first tragus point on the first side and the second tragus point on the second side is proportional to the height of the head in the sagittal plane.

8. A vehicle, characterized in that, It includes a memory and a processor, wherein the memory stores a computer program, and when the computer program is executed by the processor, it implements the method according to any one of claims 1-7.

9. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores a computer program that, when executed by one or more processors, implements the method of any one of claims 1-7.

10. A computer program product comprising a computer program / instructions, characterized in that, When the computer program / instructions are executed by the processor, they implement the method described in any one of claims 1-7.

Images