A car seat automatic adjustment system and method capable of identifying passenger body shape

Abstract

The present invention discloses an automatic adjustment system and method for automobile seats that can identify the body shape of passengers. The system uses a depth camera installed outside the car to identify the passenger's body parameters before getting on the car, including height, sitting height, leg length, arm length, etc., and simultaneously performs facial recognition to bind the passenger's body parameters and face. After the passenger gets on the car, the camera inside the car recognizes the face and obtains the corresponding seat for the passenger. The microcontroller finds the corresponding body parameters based on the passenger's face and automatically adjusts the seat's front and rear position, height, and backrest angle and other parameters to automatically adapt to the passenger's body shape. The smart seat also allows passengers to adjust the seat according to their own habits. Once set, the most comfortable seat parameters for the passenger can be saved through the cloud platform. Before the passenger gets on the car again, the system recognizes the passenger's identity through facial recognition and automatically adjusts the seat according to the most recently adjusted seat parameters.

Description

Technical Field

[0001] The present invention relates to an automatic adjustment system and method for automobile seats capable of identifying the body shape of passengers, belonging to the technical field of intelligent adjustment of automobile seats. Background Art

[0002] With the development of the automotive industry, China's car ownership continues to rise, driving demand for a safer and more comfortable car experience. Car seats are essential components that come into direct contact with the human body, supporting the weight of occupants, providing good working conditions for the driver, and creating a safe and comfortable environment for passengers. The convenience and comfort of car seats often impact the driver's field of view, riding experience, and mental state, and even their driving safety. A suitable seat not only ensures occupant comfort but also provides adequate restraint, mitigating injuries in accidents. Due to the diverse body shapes and driving habits of drivers, car seats often feature multiple position adjustments, such as fore / aft, height, and angle. However, these adjustments require manual adjustment after each occupant changes, resulting in an inefficient process. To provide a better driving experience for both drivers and passengers while enabling automatic seat adjustment, a 3D depth camera is needed to identify human body parameters.

[0003] Traditional two-dimensional vision systems provide relevant two-dimensional image data through flat photos, and then perform two-dimensional geometric analysis. However, 2D cameras cannot obtain the spatial coordinate information of objects, making it difficult to complete measurements related to object size and volume alone. In addition to obtaining two-dimensional image data from traditional cameras, three-dimensional vision systems can also obtain depth information from images, greatly improving the accuracy of three-dimensional measurements. With the help of 3D cameras, passengers are identified, human posture estimation parameters are obtained, and data such as the height and angle between the upper and lower body of the driver and passenger are calculated. Automatic seat adjustment can be completed before the passenger gets on the vehicle, improving the comfort and convenience of the ride. Currently, 3D depth cameras that can identify human body parameters in domestic and foreign research can be divided into three categories according to their principles: binocular cameras, structured light cameras, and TOF cameras.

[0004] Binocular cameras use the matching and triangulation principles of the image's inherent feature points for distance measurement. They have the advantages of low cost and can be used both indoors and outdoors. However, the official RealSense SDK software does not have a skeletal tracking function and must be used in conjunction with other software platforms, making implementation more difficult. Representative products include the Intel D400 series.

[0005] Structured light cameras actively project patterns containing coded information onto the surface of an object for feature matching. They offer advantages such as mature algorithms, ease of miniaturization, and insensitivity to object texture and lighting variations. However, they can only be used in indoor or semi-outdoor environments (light levels less than 5000 lux, i.e., on cloudy days), and their skeleton recognition solutions are not open source. Representative products include Orbbec's Astro Pro and Gemini Pro series.

[0006] TOF cameras use the time of flight of light to directly measure objects. They have the advantages of long detection distance and are not easily affected by object texture characteristics and lighting changes. The official SDK provides skeleton tracking functions, and skeleton recognition is widely used and mature. Representative products include the Kinect series launched by Microsoft.

[0007] In order to better identify the body parameters of the human body, the Kinect depth camera is used to achieve this function. Therefore, a smart car seat based on the Kinect depth camera that can combine body data analysis, face recognition and seat adjustment is urgently needed. Summary of the Invention

[0008] The purpose of this invention is to design an intelligent seat with the function of recognizing the body shape and face of the occupants, so that the occupants can obtain the most comfortable driving and riding experience, and provide practical technical support for the application research of my country's automobile safety assisted driving system.

[0009] The technical solution of the present invention:

[0010] The present invention provides an automatic adjustment system for automobile seats based on passenger body shape recognition, (such as Figure 1 The figure shows a schematic diagram of an automatic car seat adjustment system based on occupant body shape recognition, including a Kinect depth camera, an in-car camera, a car electric seat, a microcontroller, an Ubuntu operating system, vehicle-side control software, and a cloud platform. The positional connection relationship between them is as follows: the Kinect depth camera and the in-car camera are connected to the microcontroller, the Ubuntu operating system and the vehicle-side control software are installed in the microcontroller, the microcontroller is connected to the cloud platform via a network, the microcontroller is connected to the car electric seat, the vehicle-side control software transmits program execution instructions to the car electric seat, and the car electric seat controls the internal worm gear motor and ball screw motor to adjust the electric seat, and records the adjustment position through a displacement sensor to verify whether the adjustment position is accurate, and comes with manual fine-tuning to achieve automatic and manual adjustment of the car seat.

[0011] The Kinect depth camera (such as Figure 2(The figure shows the appearance of the depth camera) and uses a 3D somatosensory camera developed by Microsoft, branded Azure Kinect DK. The Azure Kinect depth camera's hardware configuration mainly includes a depth camera, a color camera, a microphone array, and an IMU inertial measurement unit. The Azure Kinect depth camera is installed near the car's door and can obtain 3D body and facial data of the driver and passengers about to get on the car. The vehicle-side control software in the microcontroller can identify the occupants' body parameters such as leg length, arm length, calf length, thigh length, and seat height based on the data sent by the depth camera. It can also record the body parameters corresponding to each face, thereby binding the body parameters of the Kinect camera to the specific face.

[0012] The in-car camera uses a wide-angle camera installed near the rearview mirror. Passenger size can only be accurately measured when the passenger is standing upright before boarding the vehicle. Passengers cannot determine their specific seats until they are on board, so the in-car camera is needed to determine the passenger's face for each seat. Facial recognition using the in-car camera can determine the passenger's identity. Combined with the facial and body parameters of the 3D depth camera, the size of each seat's occupant can be determined.

[0013] The Ubuntu operating system is installed in the microcontroller and can store occupant body shape recognition programs and face recognition programs written in Visual C++. It can also calculate input human body parameters on the vehicle, output the coordinates of key points such as the head, chest, neck, and hips, and calculate the occupant's leg length, arm length, calf length, thigh length, seat height and other body shape parameters, thereby achieving a full range of body shape recognition functions.

[0014] The microcontroller is a microcomputer system that uses a 32-bit or 64-bit embedded processor circuit. The microcontroller is installed with an operating system and application software, which can perform body shape recognition, face recognition, and send commands for the car's power seat based on data collected by the depth camera.

[0015] The electric car seat includes a ball screw motor, a worm gear motor, a position sensor, a single-chip microcomputer, and other components. The microcontroller transmits the results of occupant body parameter recognition to the electric car seat via a data interface. The single-chip microcomputer controls the movement of the ball screw motor and worm gear motor, while simultaneously reading feedback from the position sensor to adjust the seat to the most accurate position.

[0016] The worm gear motor is located on the electric seat of the car. The worm gear motor is mainly used to adjust the backrest angle of the electric seat. The backrest angle can be changed arbitrarily. The worm gear motor receives instructions from the controller to change the inclination angle of the electric seat and adjust the seat to a specified position to meet the passenger's comfortable inclination requirements.

[0017] The ball screw motor is located on the car's electric seat. The motor is placed in the horizontal and vertical areas of the electric seat, with a total of two motors. When the ball screw motor is working, the nut is connected to the electric seat that needs to perform linear reciprocating motion. The ball screw motor converts rotational motion into linear motion. There is a ball screw motor in the horizontal direction and the vertical direction respectively. The ball screw motor receives instructions from the single-chip microcomputer and drives the electric seat to adjust to the specified position in the horizontal direction and the vertical direction respectively.

[0018] The position sensor is located on the electric seat of the car. The position sensor uses the principle of magnetostriction to accurately measure the position by generating a strain pulse signal through the intersection of two different magnetic fields. Three position sensors are required. Through the lateral and vertical seat adjustments, one position sensor is used to calculate the coordinates after movement. Another position sensor is used to record the angle between the lateral and vertical directions to meet the passenger's comfortable tilt angle requirements.

[0019] Based on the above system, the present invention provides a method for automatically adjusting a car seat that can identify the body shape of an occupant. The method specifically comprises the following steps:

[0020] Step 1: Body shape parameter identification

[0021] Step S101: Connect the Kinect depth camera to the microcontroller (such as Figure 3 The figure shows the internal structure of the depth camera. Using the Kinect depth camera SDK, the height, leg length, arm length, calf length, thigh length, sitting height, etc. of the occupant are identified (e.g. Figure 4The figure shows the effect of the sensor SDK recognizing the human body), and identifies the horizontal and vertical coordinates of the key points of the occupants, as well as the longitudinal coordinates of the occupants from the depth camera, and uses the Body Tracking SDK to capture the key points of the human skeleton joints. The skeleton of the Kinect human body tracking recognition includes 32 joints, but the height, angle, front and back distance and other position adjustments of the seat are mainly related to the length of the driver's upper and lower body, arm length (determines the distance from the steering wheel and can be used to calculate the height), and the angle of the upper body. Therefore, the head (1), neck (2), chest spine (3), navel spine (4), left and right clavicles (5) (6), left and right shoulders (7) (8), left and right elbows (9) (10), left and right hands (11) (12), left and right fingertips (13) (14), left and right hips (15) (16), left and right knees (17) (18), left and right ankles (19) (20), left and right feet (21) (22) (such as Figure 5 The corresponding table of skeleton key points and serial numbers shown in the figure) shows these skeleton key points (such as Figure 6 The figure shows a 3D rendering of the human skeleton inference using the Human Tracking SDK. The captured data is queued to obtain the coordinates of the skeletal joints, and finally, points and lines are drawn in the RGB image. The microcontroller uses data collected by a depth camera mounted outside the car to identify the body parameters of the occupant before they even get in.

[0022] Step S102: The microcontroller uses the image data collected by the depth camera to perform facial recognition on the passenger who is about to get on the bus, and binds it with the passenger's body parameters recognized at the same time.

[0023] Step S103: After getting on the bus, the passenger uses facial recognition technology to determine the passenger's position; the camera inside the car recognizes the face, so that the passenger's corresponding seat can be known.

[0024] Step 2: Automatic seat adjustment

[0025] Step S201: Write down the key joint point parameter determination method (such as Figure 7 (See the flowchart of the Visual C++ program code running process shown in the figure), a Visual C++ program is compiled using the Ubuntu operating system according to the key joint parameter determination method and transmitted to the microcontroller;

[0026] The method for determining the parameters of key joints is as follows:

[0027] Methods for determining key joint parameters (such as Figure 6 The Visual C++ program code running process flow chart shown is as follows:

[0028] In the rectangular three-dimensional coordinate system, let the three-dimensional coordinates of each key point joint be P i (x i ,yi ,z i ), using the Euclidean distance calculation formula, let d(P i ,P j ) indicates that the Euclidean distance between the joint corresponding to number i and the joint corresponding to number j can be expressed as:

[0029]

[0030] Upper body height (upper_body) can be expressed as:

[0031] upper_body=d(P2,P4) (2)

[0032] The lower body (i.e. leg length) (length_left, length_right) can be expressed as:

[0033] length_left=d(P 15 ,P 17 )+d(P 17 ,P 19 ) (3)

[0034] length_right=d(P 16 ,P 18 )+d(P 18 ,P 20 ) (4)

[0035] The average value of the lower body (i.e. leg length) (lower_body) is:

[0036] lower_body=(length_left+length_right) / 2 (5)

[0037] The arm length (arm_left, arm_right) can be expressed as:

[0038] arm_left=d(P7,P9)+d(P9,P 11 )+d(P 11 ,P 13 ) (6)

[0039] arm_right=d(P8,P 10 )+d(P 10 ,P 12 )+d(P 12 ,P 14 ) (7)

[0040] The average arm length (arm_length):

[0041] arm_length=(arm_left+arm_right) / 2 (8)

[0042] The upper body inclination angle (angle_left, angle_right) can be expressed as:

[0043] angle_left=arccos{[d 2 (P5,P 15 )+d 2 (P 15 ,P 17 )-d 2 (P5,P 17 )] / (2*d(P5,P 15 )+d(P 15 ,P 17 )} (9)

[0044] angle_right=arccos{[d 2 (P6,P 16 )+d 2 (P 16 ,P 18 )-d 2 (P6,P 18 )] / (2*d(P6,P 16 )+d(P 16 ,P 18 )}(10)

[0045] Average upper body inclination angle (body_angle):

[0046] body_angle=(angle_left+angle_right) / 2 (11)

[0047] In order to further evaluate the key node values ​​calculated by the program written in Visual C++, a data set of 1000 was initially collected (including standing still, walking from far to near, or from near to far), and the relationship between different data was plotted and compared, and a simple linear fit (such as Figure 8 The corresponding relationship diagram of each data collected is shown in FIG), and the actual lower body and arm length are measured based on the key points of the skeleton, and compared with the data measured using Kinect (such as Figure 9 The relationship between the calculated value and the distance, and the comparison with the measured value are shown). Figure 9Analysis shows that the camera's primary operating range is 2000mm-4000mm. Measurements of human bodies smaller than this range are incomplete, while measurements larger than this range can lead to difficulty recognizing the human body and large errors. Even within this effective operating range, some interfering noise points can still cause errors compared to the true value. Determining the optimal measurement results within a short capture time requires further optimization. Therefore, data processing and algorithm optimization are necessary.

[0048] The data processing methods currently used are:

[0049] (1) Clustering-based outlier detection. This method is based on the following assumptions: normal data should belong to a large and dense cluster in the data, close to the nearest cluster centroid, while abnormal data does not belong to any cluster or belongs to a small and sparse cluster, far away from the nearest cluster centroid. Outliers are a by-product of clustering, so common clustering algorithms such as K-means, DBSCAN, CHAMELEON, CLIQUE, etc. can be modified to be used for outlier detection. Since the input of the data stream is dynamically changing, it makes it difficult for supervised methods to start, so unsupervised clustering methods are more suitable for outlier detection in data streams. This design uses dynamic data stream input, so the Kmeans clustering method is used. Considering that the small cluster data obtained by clustering is classified as anomalies, the two methods of selecting four feature points and two feature points are used for anomaly detection to obtain the results (such as Figure 10 The two K-means-based outlier detection results shown (the above is a schematic diagram after dimensionality reduction) are not ideal.

[0050] (2) Density-based outlier detection. The LOF algorithm is simpler and more intuitive than statistics-based and clustering-based anomaly detection algorithms. It is based on the following assumption: the density around non-outlier objects is similar to the density around their neighborhood, while the density around outlier objects is significantly different from the density around their neighborhood. By calculating and comparing the size of the density-based outlier factor LOF, it is determined whether it is a normal data point.

[0051] (3) Data filtering: Consider using Kalman filtering to process the data. Kalman filtering can estimate the past and current states of the signal and reduce the impact of measurement noise. It also has a small memory footprint and high speed. The measured values ​​can be used to further correct the state and obtain the true value.

[0052] (4) Numerical compensation: There is still a gap between the human body parameters required for seat adjustment and the calculated positions of the key points of the skeleton. At the same time, the accuracy of the depth frame collected by the Kinect camera is related to the distance between the Kinect and the object. The closer the distance to the object to be scanned, the higher the accuracy of the scanned data. The calculated value can be compensated and corrected based on the actual measurement value.

[0053] Step S202: The microcontroller sends the identified body shape data of each passenger, including height, leg length, arm length, calf length, thigh length, seat height, etc., to the single-chip microcomputer in each seat of the car electric seat through the communication interface;

[0054] Step S203: After receiving the body shape data of the occupant, the microcontroller in the electric car seat calculates the optimal parameters of the seat, including horizontal position, height, and seat back angle, based on the body shape data and a comparison table of electric car seat parameters;

[0055] Step S204: Sending instructions to the position adjustment mechanisms such as the ball screw motor and the worm gear motor to adjust the horizontal position, height and backrest angle of the car power seat to the optimal parameters;

[0056] Step S205: Using the ball screw motor to adjust the horizontal position and height of the electric seat of the car;

[0057] Step S206: using the worm gear motor to adjust the backrest angle of the electric car seat;

[0058] Step S207: using a displacement sensor to feed back the horizontal position and height of the electric seat adjusted by the ball screw motor, and using an angle sensor to feed back the backrest angle of the electric seat;

[0059] Step S208: the single chip microcomputer inside the electric car seat reports the current parameters of the electric car seat to the microcontroller via the communication interface in real time.

[0060] Step 3: Manually fine-tune the seat

[0061] Smart seats also allow passengers to adjust the seats according to their own habits. There are some manual adjustment buttons on the car's electric seats, so passengers can fine-tune the seat parameters according to their needs.

[0062] Step 4: Database parameter storage

[0063] The identified occupant's height, seat height, leg length, arm length, and other parameters, as well as the adjusted seat fore-aft position, height, and backrest angle, are recorded and associated with the occupant's identity as determined by facial recognition. This information is then reported to the cloud platform via a microcontroller and recorded in the cloud platform's database. This way, when this occupant returns to the vehicle, the system will automatically adjust the seat based on the most recently adjusted parameters after facial recognition.

[0064] Beneficial effects of the present invention:

[0065] 1. Key body parameter measurement methods using depth cameras, such as density-based outlier detection, data filtering, and numerical compensation, can improve the accuracy of key body parameter measurement and are more conducive to the accuracy of depth cameras in identifying key body joints;

[0066] 2. The automatic adjustment system based on the depth camera saves the driver and passengers the time of manually adjusting the seats, providing convenience for the driver and passengers;

[0067] 3. The depth camera-based automatic adjustment system improves the accuracy of seat adjustment, allowing drivers and passengers to achieve a more comfortable sitting posture, especially helping the driver to obtain a better field of view, thereby ensuring greater driving safety;

[0068] 4. The automatic adjustment system based on depth cameras is conducive to the popularization of shared cars, thereby driving the rapid development of the automotive industry; BRIEF DESCRIPTION OF THE DRAWINGS

[0069] Figure 1 : A schematic diagram of an automatic car seat adjustment system based on occupant body shape recognition;

[0070] Figure 2 : Schematic diagram of camera appearance;

[0071] Figure 3 : Schematic diagram of the internal structure of the camera;

[0072] Figure 4 : Image of the sensor SDK recognizing the human body;

[0073] Figure 5 : Correspondence table of skeleton key points and serial numbers;

[0074] Figure 6 : Human body tracking SDK human skeleton inference 3D rendering;

[0075] Figure 7 :Visual C++ program code running process flow chart;

[0076] Figure 8 : Corresponding relationship diagram of collected data;

[0077] Figure 9 : The relationship between the calculated value and the distance, and the comparison with the measured value;

[0078] Figure 10 : Two outlier detection results based on K-means (the above is a schematic diagram after dimensionality reduction); DETAILED DESCRIPTION

[0079] The implementation of the present invention mainly includes the following four aspects:

[0080] 1. Body measurement recognition. Using a depth camera mounted on the vehicle's exterior, the occupant's body measurements, including height, seat height, leg length, and arm length, are determined before boarding. Facial recognition is also performed simultaneously, linking the occupant's body measurements to their face. After boarding, the in-car camera recognizes their face and determines their corresponding seat.

[0081] 2. Automatic seat adjustment. The microcontroller identifies the corresponding body parameters based on the face of each seat occupant and automatically adjusts the seat's fore-and-aft position, height, and backrest angle to accommodate the occupant's body shape. The driver's and front passenger seats can each accommodate only one occupant, eliminating the need for facial matching. The seat parameters can be adjusted directly based on the body parameters detected by the exterior depth camera. The driver's and front passenger seats automatically adjust before the occupants board the vehicle because the depth camera identifies the occupant's body shape and determines whether the occupant should sit in the driver's or front passenger seat. However, the rear seats cannot determine the occupant's specific position. Only after the passenger sits down can the system determine the occupant's specific seat through facial recognition and then automatically adjust the seat using the body parameters detected before the occupant boarded the vehicle.

[0082] 3. Manual seat fine-tuning. The smart seat also allows passengers to adjust the seat according to their own habits. Once set, the most comfortable seat parameters for the passenger can be saved through the cloud platform. Then, when the passenger gets on the vehicle again, the system will automatically adjust the seat according to the most recently adjusted parameters after recognizing the passenger's identity through facial recognition.

[0083] 4. Database parameter storage. The identified occupant's height, seat height, leg length, arm length, and other parameters, as well as the adjusted seat fore-aft position, height, and backrest angle, are recorded and bound to the occupant's identity obtained through facial recognition and stored in the cloud platform database.

[0084] The present invention will be further described below with reference to the accompanying drawings.

[0085] The present invention provides an automatic adjustment system for automobile seats based on passenger body shape recognition, (such as Figure 1The figure shows a schematic diagram of an automatic car seat adjustment system based on occupant body shape recognition, which includes a Kinect depth camera, an in-car camera, a car electric seat, a microcontroller, an Ubuntu operating system, vehicle-side control software, and a cloud platform. The positional connection relationship between them is as follows: the Kinect depth camera and the in-car camera are connected to the microcontroller, the Ubuntu operating system and the vehicle-side control software are installed in the microcontroller, the microcontroller is connected to the cloud platform via the network, the microcontroller is connected to the car electric seat, the vehicle-side control software transmits program execution instructions to the car electric seat, and the car electric seat controls the internal worm gear motor and ball screw motor to adjust the electric seat, and records the adjustment position through a displacement sensor to verify whether the adjustment position is accurate, and comes with manual fine-tuning to achieve automatic and manual adjustment of the car seat.

[0086] The Kinect depth camera (such as Figure 2 (The figure shows the appearance of the depth camera) and uses a 3D somatosensory camera developed by Microsoft, branded Azure Kinect DK. The Azure Kinect depth camera's hardware configuration mainly includes a depth camera, color camera, microphone array, and IMU inertial measurement unit. The Azure Kinect depth camera is installed near the car's door and can obtain 3D body and facial data of the driver and passengers about to get on the car. The vehicle-side control software in the microcontroller can use the data sent by the depth camera to identify the occupants' body parameters such as leg length, arm length, calf length, thigh length, and seat height. It can also record the body parameters corresponding to each face, thereby binding the body parameters of the Kinect camera to the specific face.

[0087] The in-car camera uses a wide-angle camera installed near the rearview mirror. Passenger size can only be accurately measured when the passenger is standing upright before boarding the vehicle. Passengers cannot determine their specific seats until they are on board, so the in-car camera is needed to determine the passenger's face for each seat. Facial recognition using the in-car camera can determine the passenger's identity. Combined with the facial and body parameters of the 3D depth camera, the size of each seat's occupant can be determined.

[0088] The Ubuntu operating system is installed in the microcontroller. Compared with the Windows operating system, it is simpler to operate and easier to use. It can store passenger body recognition programs and face recognition programs written in Visual C++, and can calculate the input human body parameters on the vehicle, output the coordinates of key points such as the head, chest, neck, and hips, and calculate the passenger's leg length, arm length, calf length, thigh length, sitting height and other body parameters, thereby achieving a comprehensive body recognition function.

[0089] The microcontroller is a microcomputer system, typically a 32-bit or 64-bit embedded processor circuit. It contains an operating system and application software that can perform body shape and face recognition based on data collected by the depth camera, as well as send commands for the car's power seats.

[0090] The electric car seat includes a ball screw motor, a worm gear motor, a position sensor, a single-chip microcomputer, and other components. The microcontroller transmits the results of occupant body parameter recognition to the electric car seat via a data interface. The single-chip microcomputer controls the movement of the ball screw motor and worm gear motor, while simultaneously reading feedback from the position sensor to adjust the seat to the most accurate position.

[0091] The worm gear reduction motor is located on the electric seat of the car. The worm gear motor is mainly used for adjusting the backrest angle of the electric seat. The backrest angle can be changed arbitrarily. The worm gear reduction motor receives instructions from the controller to change the inclination angle of the electric seat and adjust the seat to a specified position to meet the passenger's comfortable inclination requirements.

[0092] The ball screw motor is located on the electric seat of the car. This motor should be placed in the horizontal and vertical areas of the electric seat, with two motors in total. When the ball screw motor is working, the nut is connected to the electric seat that needs to perform linear reciprocating motion. The ball screw motor converts rotational motion into linear motion. There is a ball screw motor in the horizontal direction and the vertical direction respectively. The ball screw motor receives instructions from the single-chip microcomputer and drives the electric seat to adjust to the specified position in the horizontal direction and the vertical direction respectively.

[0093] The position sensor is located on the electric seat of the car. The position sensor uses the principle of magnetostriction to accurately measure the position by generating a strain pulse signal through the intersection of two different magnetic fields. Three position sensors are required. Through the lateral and vertical seat adjustments, one position sensor is used to calculate the coordinates after movement. Another position sensor is used to record the angle between the lateral and vertical directions to meet the passenger's comfortable tilt angle requirements.

[0094] The present invention provides a method for automatically adjusting a car seat capable of identifying the body shape of an occupant. The method specifically comprises the following steps:

[0095] Step 1: Body shape parameter identification

[0096] Step S101: Connect the Kinect depth camera to the microcontroller (such as Figure 3 The figure shows the internal structure of the depth camera. Using the Kinect depth camera SDK, the height, leg length, arm length, calf length, thigh length, sitting height, etc. of the occupant are identified (e.g. Figure 4The figure shows the effect of the sensor SDK recognizing the human body), and identifies the horizontal and vertical coordinates of the key points of the occupants, as well as the longitudinal coordinates of the occupants from the depth camera, and uses the Body Tracking SDK to capture the key points of the human skeleton joints. The skeleton of the Kinect human body tracking recognition includes 32 joints, but the height, angle, front and back distance and other position adjustments of the seat are mainly related to the length of the driver's upper and lower body, arm length (determines the distance from the steering wheel and can be used to calculate the height), and the angle of the upper body. Therefore, the head (1), neck (2), chest spine (3), navel spine (4), left and right clavicles (5) (6), left and right shoulders (7) (8), left and right elbows (9) (10), left and right hands (11) (12), left and right fingertips (13) (14), left and right hips (15) (16), left and right knees (17) (18), left and right ankles (19) (20), left and right feet (21) (22) (such as Figure 5 The corresponding table of skeleton key points and serial numbers shown in the figure) shows these skeleton key points (such as Figure 6 The figure shows a 3D rendering of the human skeleton inference using the Human Tracking SDK. The captured data is queued to obtain the coordinates of the skeletal joints, and finally, points and lines are drawn in the RGB image. The microcontroller uses data collected by a depth camera mounted outside the car to identify the body parameters of the occupant before they even get in.

[0097] Step S102: The microcontroller uses the image data collected by the depth camera to perform facial recognition on the passenger who is about to get on the bus, and binds it with the passenger's body parameters recognized at the same time.

[0098] Step S103: After getting on the bus, the passenger uses facial recognition technology to determine the passenger's position; the camera inside the car recognizes the face, so that the passenger's corresponding seat can be known.

[0099] Step 2: Automatic seat adjustment

[0100] Step S201: Write a key joint parameter measurement method, use the Ubuntu operating system to compile a VC++ program based on the key joint parameter measurement method and transmit it to the microcontroller;

[0101] Methods for determining key joint parameters (such as Figure 7 The Visual C++ program code running process flow chart shown is as follows:

[0102] In the rectangular three-dimensional coordinate system, let the three-dimensional coordinates of each key point joint be P i (x i ,y i ,z i ), using the Euclidean distance calculation formula, let d(P i ,P j) indicates that the Euclidean distance between the joint corresponding to number i and the joint corresponding to number j can be expressed as:

[0103]

[0104] Upper body height (upper_body) can be expressed as:

[0105] upper_body=d(P2,P4) (2)

[0106] The lower body (i.e. leg length) (length_left, length_right) can be expressed as:

[0107] length_left=d(P 15 ,P 17 )+d(P 17 ,P 19 ) (3)

[0108] length_right=d(P 16 ,P 18 )+d(P 18 ,P 20 ) (4)

[0109] The average value of the lower body (i.e. leg length) (lower_body) is:

[0110] lower_body=(length_left+length_right) / 2 (5)

[0111] The arm length (arm_left, arm_right) can be expressed as:

[0112] arm_left=d(P7,P9)+d(P9,P 11 )+d(P 11 ,P 13 ) (6)

[0113] arm_right=d(P8,P 10 )+d(P 10 ,P 12 )+d(P 12 ,P 14 ) (7)

[0114] The average arm length (arm_length):

[0115] arm_length=(arm_left+arm_right) / 2 (8)

[0116] The upper body inclination angle (angle_left, angle_right) can be expressed as:

[0117] angle_left=arccos{[d 2 (P5,P 15 )+d 2 (P 15 ,P 17 )-d 2 (P5,P 17 )] / (2*d(P5,P 15 )+d(P 15 ,P 17 )} (9)

[0118] angle_right=arccos{[d 2 (P6,P 16 )+d 2 (P 16 ,P 18 )-d 2 (P6,P 18 )] / (2*d(P6,P 16 )+d(P 16 ,P 18 )}(10)

[0119] Average upper body inclination angle (body_angle):

[0120] body_angle=(angle_left+angle_right) / 2 (11)

[0121] In order to further evaluate the key node values ​​calculated by the program written in Visual C++, a data set of 1000 was initially collected (including standing still, walking from far to near, or from near to far), and the relationship between different data was plotted and compared, and a simple linear fit (such as Figure 8 The corresponding relationship diagram of each data collected is shown in FIG), and the actual lower body and arm length are measured based on the key points of the skeleton, and compared with the data measured using Kinect (such as Figure 9 The relationship between the calculated value and the distance, and the comparison with the measured value are shown). Figure 9 Analysis shows that the camera's primary operating range is 2000mm-4000mm. Measurements of human bodies smaller than this range are incomplete, while measurements larger than this range can lead to difficulty recognizing the human body and large errors. Even within this effective operating range, some interfering noise points can still cause errors compared to the true value. Determining the optimal measurement results within a short capture time requires further optimization. Therefore, data processing and algorithm optimization are necessary.

[0122] The data processing methods currently being considered include:

[0123] (1) Clustering-based outlier detection. This method is based on the following assumptions: normal data should belong to a large and dense cluster in the data, close to the nearest cluster centroid, while abnormal data does not belong to any cluster or belongs to a small and sparse cluster, far away from the nearest cluster centroid. Outliers are a by-product of clustering, so common clustering algorithms such as K-means, DBSCAN, CHAMELEON, CLIQUE, etc. can be modified to be used for outlier detection. Since the input of the data stream is dynamically changing, it makes it difficult for supervised methods to start, so unsupervised clustering methods are more suitable for outlier detection in data streams. This design uses dynamic data stream input, so the Kmeans clustering method is used. Considering that the small cluster data obtained by clustering is classified as anomalies, the two methods of selecting four feature points and two feature points are used for anomaly detection to obtain the results (such as Figure 10 The two K-means-based outlier detection results shown (the above is a schematic diagram after dimensionality reduction) are not ideal.

[0124] (2) Density-based outlier detection. The LOF algorithm is simpler and more intuitive than statistics-based and clustering-based anomaly detection algorithms. It is based on the following assumption: the density around non-outlier objects is similar to the density around their neighborhood, while the density around outlier objects is significantly different from the density around their neighborhood. By calculating and comparing the size of the density-based outlier factor LOF, it is determined whether it is a normal data point.

[0125] (3) Data filtering: Consider using Kalman filtering to process the data. Kalman filtering can estimate the past and current states of the signal and reduce the impact of measurement noise. It also has a small memory footprint and high speed. The measured values ​​can be used to further correct the state and obtain the true value.

[0126] (4) Numerical compensation: There is still a gap between the human body parameters required for seat adjustment and the calculated positions of the key points of the skeleton. At the same time, the accuracy of the depth frame collected by the Kinect camera is related to the distance between the Kinect and the object. The closer the distance to the object to be scanned, the higher the accuracy of the scanned data. The calculated value can be compensated and corrected based on the actual measurement value.

[0127] Step S202: The microcontroller sends the identified body shape data of each passenger, including height, leg length, arm length, calf length, thigh length, seat height, etc., to the single-chip microcomputer in each seat of the car electric seat through the communication interface;

[0128] Step S203: After receiving the body shape data of the occupant, the microcontroller in the electric car seat calculates the optimal parameters of the seat, including horizontal position, height, and seat back angle, based on the body shape data and a comparison table of electric car seat parameters;

[0129] Step S204: Sending instructions to the position adjustment mechanisms such as the ball screw motor and the worm gear motor to adjust the horizontal position, height and backrest angle of the car power seat to the optimal parameters;

[0130] Step S205: Using the ball screw motor to adjust the horizontal position and height of the electric seat of the car;

[0131] Step S206: using the worm gear motor to adjust the backrest angle of the electric car seat;

[0132] Step S207: using a displacement sensor to feed back the horizontal position and height of the electric seat adjusted by the ball screw motor, and using an angle sensor to feed back the backrest angle of the electric seat;

[0133] Step S208: the single chip microcomputer inside the electric car seat reports the current parameters of the electric car seat to the microcontroller via the communication interface in real time.

[0134] Step 3: Manually fine-tune the seat

[0135] Smart seats also allow passengers to adjust the seats according to their own habits. There are some manual adjustment buttons on the car's electric seats, so passengers can fine-tune the seat parameters according to their needs.

[0136] Step 4: Database parameter storage

[0137] The identified occupant's height, seat height, leg length, arm length, and other parameters, as well as the adjusted seat fore-aft position, height, and backrest angle, are recorded and associated with the occupant's identity as determined by facial recognition. This information is then reported to the cloud platform via a microcontroller and recorded in the cloud platform's database. This way, when this occupant returns to the vehicle, the system will automatically adjust the seat based on the most recently adjusted parameters after facial recognition.

[0138] The series of detailed descriptions listed above are only specific descriptions of feasible implementation methods of the present invention. They are not intended to limit the scope of protection of the present invention. Any equivalent methods or changes that do not deviate from the technology of the present invention should be included in the scope of protection of the present invention.

Claims

1. A method for automatically adjusting a car seat capable of recognizing the body shape of an occupant, characterized in that: The steps include: Step S1: Body shape parameter identification; details are as follows Step S101: Connect the depth camera to the microcontroller and use the depth camera SDK to identify the occupant's height, leg length, arm length, calf length, thigh length, and seat height. Also, identify the horizontal and vertical coordinates of the occupant's key points, as well as the longitudinal coordinates of the occupant from the depth camera. Use Body Tracking SDK captures the key points of human skeleton joints. The skeleton of the human body tracking recognition of the depth camera includes 32 joints. However, the height, angle, and front-to-back distance position adjustment of the seat are related to the length of the upper and lower body, arm length, and upper body angle of the driver. Therefore, the key points of the skeleton features such as the head (1), neck (2), chest spine (3), navel spine (4), left and right clavicles (5)(6), left and right shoulders (7)(8), left and right elbows (9)(10), left and right hands (11)(12), left and right fingertips (13)(14), left and right hips (15)(16), left and right knees (17)(18), left and right ankles (19)(20), and left and right feet (21)(22) are captured, and the captured results are queued to obtain the coordinate points of the skeleton joints. Finally, the points and lines are drawn in the RGB image. Step S102: The microcontroller uses the image data collected by the depth camera to perform facial recognition on the passenger about to board the vehicle, and binds the facial data with the passenger's body parameters recognized at the same time; Step S103: After boarding the vehicle, the passenger uses facial recognition technology to determine the passenger's location; the camera inside the vehicle recognizes the face and obtains the passenger's corresponding seat; Step S2: Automatic seat adjustment; details are as follows Step S201: Design a key joint parameter measurement method, use the Ubuntu operating system to compile a program based on the key joint parameter measurement method and transmit it to the microcontroller; In S201, the method for determining the key joint parameters is as follows: In the rectangular three-dimensional coordinate system, let the three-dimensional coordinates of each key point joint be P i (x i ,y i ,z i ), using the Euclidean distance calculation formula, let d(P i ,P j ) indicates that the Euclidean distance between the joint corresponding to number i and the joint corresponding to number j can be expressed as: Upper body height (upper_body) can be expressed as: upper_body=d(P2,P4) (2) The lower body (i.e. leg length) (length_left, length_right) can be expressed as: length_left=d(P 15 ,P 17 )+d(P 17 ,P 19 ) (3) length_right=d(P 16 ,P 18 )+d(P 18 ,P 20 ) (4) The average value of the lower body (i.e. leg length) (lower_body) is: lower_body=(length_left+length_right) / 2 (5) The arm length (arm_left, arm_right) can be expressed as: arm_left=d(P7,P9)+d(P9,P 11 )+d(P 11 ,P 13 ) (6) arm_right=d(P8,P 10 )+d(P 10 ,P 12 )+d(P 12 ,P 14 ) (7) The average arm length (arm_length): arm_length=(arm_left+arm_right) / 2 (8) The upper body inclination angle (angle_left, angle_right) can be expressed as: angle_left=arccos{[d 2 (P5,P 15 )+d 2 (P 15 ,P 17 )-d 2 (P5,P 17 )] / (2*d(P5,P 15 )+d(P 15 ,P 17 )}(9) angle_right=arccos{[d 2 (P6,P 16 )+d 2 (P 16 ,P 18 )-d 2 (P6,P 18 )] / (2*d(P6,P 16 )+d(P 16 ,P 18 )}(10) Average upper body inclination angle (body_angle): body_angle=(angle_left+angle_right) / 2 (11) To evaluate the key node values ​​calculated by the program, we initially collected a data set of 1,000, including standing still, walking from far to near, and walking from near to far. We plotted and compared the relationships between the different data points and performed simple linear fits. We also measured the actual lower body and arm lengths based on the skeletal key points and compared them with the data measured using the depth camera. This data needed to be processed and optimized, and the data processing methods used included: (1) Clustering-based outlier detection. This method is based on the following assumptions: normal data should belong to a large and dense cluster in the data, close to the nearest cluster centroid, while abnormal data does not belong to any cluster or belongs to a small and sparse cluster, far away from the nearest cluster centroid. Outliers are a by-product of clustering. Since the input of the data stream is dynamically changing, this makes it difficult for supervised methods to start. Therefore, unsupervised clustering methods are more suitable for outlier detection in data streams. In this design, the data stream input is dynamic, so the Kmeans clustering method is used. Considering that the small cluster data obtained by clustering is classified as anomalies, the two methods of selecting four feature points and two feature points are used for anomaly detection to obtain the results; (2) Density-based outlier detection: the density around non-outlier objects is similar to the density around their neighborhood, while the density around outlier objects is significantly different from the density around their neighborhood. By calculating and comparing the size of the density-based outlier factor (LOF), we can determine whether it is a normal data point. (3) Data filtering: Kalman filtering can be used to estimate the past and current states of the signal, reducing the impact of measurement noise. It also has a small memory footprint and high speed. The measured readings can be used to further correct the state and obtain the true value. (4) Numerical compensation: Since there is still a gap between the human body parameters required for seat adjustment and the calculated positions of the key points of the skeleton, and the depth frame accuracy captured by the depth camera is related to the distance between the depth camera and the object, the closer the distance to the object to be scanned, the higher the accuracy of the scanned data. The calculated values ​​can be compensated and corrected based on the actual measured values. Step S202: The microcontroller sends the identified body shape data of each passenger, including height, leg length, arm length, calf length, thigh length, and seat height, to the single-chip microcomputer in each seat of the car's electric seat through the communication interface; Step S203: After receiving the body shape data of the occupant, the single chip microcomputer in the electric car seat calculates the optimal parameters of the seat, including horizontal position, height and seat back angle, based on the body shape data and a comparison table of electric car seat parameters; Step S204: Sending instructions to the ball screw motor and worm gear motor position adjustment mechanism to adjust the horizontal position, height and seat back angle of the car power seat to optimal parameters; Step S205: Using the ball screw motor to adjust the horizontal position and height of the electric seat of the car; Step S206: using the worm gear motor to adjust the backrest angle of the electric car seat; Step S207: using a displacement sensor to feed back the horizontal position and height of the electric seat adjusted by the ball screw motor, and using an angle sensor to feed back the backrest angle of the electric seat; Step S208: The single chip microcomputer inside the electric seat of the car reports the current parameters of the electric seat of the car to the microcontroller via the communication interface in real time, forming a closed-loop control.

2. The method for automatically adjusting a car seat capable of recognizing the body shape of an occupant according to claim 1, characterized in that: The step S3 is also included: manual fine adjustment of the seat, which is as follows: Smart seats also allow passengers to adjust the seats according to their own habits. Manual adjustment buttons are set on the car's electric seats, so passengers can fine-tune the seat parameters according to their needs.

3. The method for automatically adjusting a car seat capable of recognizing an occupant's body shape according to claim 1, characterized in that: The step S4 is also included: storing parameters in the database, specifically as follows: The height, sitting height, leg length, arm length parameters of the identified occupant and the adjusted seat fore-and-aft position, height and backrest angle parameters are recorded, bound to the occupant's identity identified by face recognition, reported to the cloud platform through the microcontroller, and recorded in the cloud platform database; when this occupant gets on the vehicle again, the identity of this occupant is recognized by face recognition, and the system automatically adjusts the seat according to the most recently adjusted seat parameters.

Images