A method and system for recognizing a sitting posture of a car seat based on machine vision

Abstract

The application provides a kind of based on machine vision's automobile seat riding posture recognition method and system, it is related to automobile seat technical field, including: real-time acquisition pressure data and pressure distribution characteristics, judge seat passenger existence;Passenger image data acquisition is carried out;Passenger image data, pressure data and pressure distribution characteristics are pretreated and feature extraction is carried out to analyze passenger type;Passenger posture recognition is carried out;Determine the danger level of current riding posture;According to the dangerous analysis result, the corresponding safety response decision is generated.The application realizes accurate judgment and dynamic monitoring to the existence of seat passenger through the multidimensional fusion perception of pressure data and image data, improves the system operation efficiency;Through three-level response mechanism and active intervention strategy, the warning and correction mode can be dynamically adjusted according to the danger level, which significantly improves the vehicle riding safety and effectively reduces the driving risk caused by bad posture.

Description

Technical Field

[0001] This invention relates to the field of automotive seat technology, and in particular to a method and system for recognizing automotive seat postures based on machine vision. Background Technology

[0002] With the rapid development of the automotive industry, especially the ever-changing technologies of intelligent vehicles and autonomous driving, automotive seat posture recognition technology has emerged and is becoming increasingly important.

[0003] From a safety perspective, accurately identifying passenger posture is crucial for ensuring occupant safety. During vehicle operation, different passenger postures significantly impact the protective effectiveness of restraint systems such as airbags and seatbelts. For example, when the system detects that an occupant is in a dangerous posture such as leaning forward or sideways, it can adjust the airbag deployment strategy in advance, and the seatbelts can automatically pretension according to the posture, ensuring that the restraint system provides optimal protection at the moment of an accident, thereby effectively reducing the risk of occupant injury.

[0004] In terms of comfort and personalized service, posture recognition technology also has broad application prospects. Everyone's body characteristics and sitting habits vary greatly. By recognizing posture, car seats can automatically adjust to the position and angle that best fits the occupant's body curves, providing just the right support and greatly alleviating fatigue during long drives. Furthermore, this technology can also work in conjunction with the vehicle's air conditioning and entertainment systems. For example, when the system determines that the occupant is in a relaxed posture, it can appropriately lower the air conditioning temperature and play soothing music, creating a more comfortable in-car environment.

[0005] Conventional car seat posture recognition methods often rely solely on pressure sensors to determine occupant presence and posture. This results in an incomplete understanding of occupant status and susceptibility to errors from single data points, affecting accuracy. Furthermore, the lack of occupant type identification makes posture recognition and hazard assessment unspecific and incompatible with the actual situations of different occupant types. Hazard assessments fail to consider the impact of dynamic factors such as vehicle movement and seatbelt usage, and lack specific response mechanisms for emergency situations, leading to insufficient real-time performance and accuracy in hazard level assessment. Moreover, safety response strategies often employ fixed warning methods, lacking a dynamically adjusted tiered response mechanism based on hazard level, thus failing to effectively reduce driving risks associated with poor posture.

[0006] To address the shortcomings of the existing technology, this technical solution proposes a machine vision-based method and system for recognizing the posture of a car seat. Summary of the Invention

[0007] This invention provides a machine vision-based method and system for recognizing the posture of a car seat, in order to overcome the deficiencies in the prior art.

[0008] On one hand, the present invention provides a machine vision-based method for recognizing the posture of a car seat, comprising:

[0009] S1: Real-time acquisition of pressure data and pressure distribution characteristics; by setting pressure thresholds and combining pressure distribution characteristics, determine the presence of seat occupants and output the occupant assessment result.

[0010] S2: Use the occupant assessment result as a trigger signal to acquire occupant image data;

[0011] S3: Preprocess and extract features from occupant image data to obtain image feature data; and extract features from pressure data and pressure distribution characteristics to obtain pressure feature data;

[0012] S4: Analyze the occupant type based on image feature data and pressure feature data, and output the occupant type analysis results;

[0013] S5: Perform feature fusion on image feature data and pressure feature data, and output the fused feature data; input the fused feature data into the pre-trained sitting posture recognition model, combine it with the occupant type analysis results, perform occupant sitting posture recognition, and output the sitting posture recognition result;

[0014] S6: Combining the occupant type analysis results and the sitting posture recognition results, compare and analyze the risk level of different sitting postures under the corresponding occupant type according to the risk level judgment criteria, determine the risk level of the current sitting posture, and output the risk analysis results;

[0015] S7: Based on the hazard analysis results, generate corresponding safety response decisions.

[0016] According to the machine vision-based method for recognizing the posture of a car seat provided by the present invention, step S1, the step of determining the presence of a seat occupant, includes:

[0017] S11: Real-time acquisition of pressure values ​​from each sensor via an array of pressure sensors deployed on the seat surface;

[0018] S12: Extract pressure distribution features based on pressure values;

[0019] S13: Set pressure threshold conditions, and determine the presence of seat occupants based on the pressure value, and output the occupant determination result.

[0020] According to the machine vision-based method for recognizing the posture of a car seat provided by the present invention, step S2, the step of acquiring occupant image data, includes:

[0021] S21: When the occupant assessment result indicates that there are occupants, the in-vehicle image acquisition device is triggered;

[0022] S22: Simultaneously acquire RGB and depth images of the upper body of the occupant, with the acquisition frequency consistent with the acquisition pressure data;

[0023] S23: If the occupant determination result is no occupants, the image acquisition device is in sleep mode.

[0024] According to the machine vision-based method for recognizing car seat postures provided by the present invention, step S3, which involves preprocessing and extracting features from the occupant image data, includes:

[0025] S31: Preprocess the RGB image and depth image, and output the preprocessed image;

[0026] S32: Based on the preprocessed image, the human pose estimation algorithm is used to extract the coordinates of key skeletal points, limb contour features and relative positional relationships of body parts of the occupant, and output preliminary image feature data.

[0027] S33: Perform three-dimensional information analysis on the depth image, combine it with the preliminary image feature data, obtain the spatial depth coordinates and limb bending angle features of each part of the upper body of the occupant, and obtain the image feature data.

[0028] According to the machine vision-based method for recognizing car seat postures provided by the present invention, step S3, the step of extracting features from pressure data and pressure distribution features, includes:

[0029] S34: Based on the pressure value, calculate the pressure center coordinates, pressure distribution entropy, and pressure gradient change rate, and output the overall pressure data;

[0030] S35: The pressure values ​​of the pressure sensor array are divided into regions using a clustering algorithm. The pressure peak value, pressure duration and pressure change frequency of each region are extracted, and the regional pressure data is output.

[0031] S36: Normalize the overall pressure data and regional pressure data, and then fuse them to obtain pressure characteristic data.

[0032] According to the machine vision-based method for recognizing car seat postures provided by the present invention, step S34, the calculation of overall pressure data includes:

[0033] S341: Calculate the pressure center coordinates using a weighted average based on the pressure value and corresponding coordinates;

[0034] S342: Based on the pressure value, calculate the proportion of each sensor's pressure value to the total pressure value, and calculate the pressure distribution entropy using the information entropy method;

[0035] S343: Based on the pressure values ​​of each sensor and the preset zones of the seat, calculate the total pressure of all sensors in each zone, calculate the percentage of pressure in each zone to the total pressure, and output the pressure percentage of each zone.

[0036] S344: Calculate the rate of change of pressure gradient based on pressure gradient data from multiple acquisition cycles.

[0037] According to the machine vision-based method for recognizing car seat postures provided by the present invention, step S4, the step of analyzing the occupant type, includes:

[0038] S41: Determine the height range and body shape characteristics of the occupants based on image feature data;

[0039] S42: Combine pressure characteristic data to determine the occupant's weight range;

[0040] S43: Input the height range, body shape characteristics, and weight range into the pre-trained classification model, and output the passenger type analysis results.

[0041] According to the machine vision-based method for recognizing car seat postures provided by the present invention, step S6, determining the danger level of the current seating posture, includes:

[0042] S61: Preset basic hazard level threshold matrix. The elements in the basic hazard level threshold matrix are the hazard values ​​corresponding to the combination of different occupant types and sitting postures.

[0043] S62: Real-time acquisition of vehicle driving status data and seat belt status signals;

[0044] S63: Based on vehicle driving status data and seat belt status signals, the vehicle dynamic parameters and occupant posture parameters are input into the membership function through a fuzzy logic controller to dynamically correct the basic hazard level threshold.

[0045] S64: When an emergency situation is detected, the danger level jump mechanism is automatically triggered, forcibly increasing the danger level of the current posture.

[0046] According to the machine vision-based method for recognizing car seat postures provided by the present invention, step S7, the step of generating corresponding safety response decisions, includes:

[0047] S71: Based on the results of hazard analysis, response levels are divided into three levels: Level 1, Level 2, and Level 3;

[0048] S72: In Level 1 response, a posture correction prompt is displayed on the in-vehicle central control screen, accompanied by a gentle prompt tone;

[0049] S73: In the second-level response, in addition to the first-level response measures, activate the seat vibration feedback and record the current posture data;

[0050] S74: In Level 3 response, the active intervention mechanism is triggered. The active intervention mechanism includes: if the vehicle is traveling at low speed, it will automatically issue a deceleration reminder and suggest stopping to adjust; if the vehicle is traveling at high speed, with the occupant's confirmation, it will make slight posture correction through the seat electric adjustment device and send a warning signal to the driver's seat at the same time.

[0051] S75: Based on the occupant's response to corrective cues, optimize the trigger threshold and execution method of subsequent coping strategies through reinforcement learning.

[0052] This invention also provides a machine vision-based automotive seat posture recognition system, comprising:

[0053] Pressure sensor array for real-time pressure data acquisition;

[0054] The pressure analysis module is used to obtain pressure distribution characteristics based on pressure data, determine the presence of seat occupants by setting pressure thresholds, and output the occupant judgment results; and extract features from pressure data and pressure distribution characteristics to obtain pressure feature data.

[0055] The image acquisition module is used to acquire occupant image data based on the occupant assessment results.

[0056] The image processing module is used to preprocess and extract features from occupant image data to obtain image feature data;

[0057] The occupant type analysis module is used to analyze occupant type based on image feature data and pressure feature data, and output the occupant type analysis results;

[0058] The posture recognition module is used to fuse image feature data and pressure feature data, and output the fused feature data. The fused feature data is then input into a pre-trained posture recognition model, and combined with the occupant type analysis results, the occupant's posture is recognized, and the posture recognition result is output.

[0059] The hazard level classification module is used to combine the occupant type analysis results and the sitting posture recognition results, compare and analyze the hazard level of different sitting postures under the corresponding occupant type according to the hazard level judgment criteria, determine the hazard level of the current sitting posture, and output the hazard analysis results.

[0060] The safety response decision generation module is used to generate corresponding safety response decisions based on the results of the hazard analysis.

[0061] This invention provides a machine vision-based method and system for recognizing the posture of car seat occupants. Through multi-dimensional fusion perception of pressure and image data, it achieves accurate judgment and dynamic monitoring of the occupant's presence, effectively reducing energy consumption from invalid image acquisition and improving system efficiency. By comprehensively judging occupant type based on height, body shape, and weight, posture recognition becomes more targeted. Employing a three-level response mechanism and proactive intervention strategy, it dynamically adjusts warning and correction methods according to the level of danger. Combined with reinforcement learning for continuous strategy optimization, it significantly improves vehicle passenger safety, providing personalized and intelligent safety protection for occupants and effectively reducing driving risks caused by poor posture. Attached Figure Description

[0062] To more clearly illustrate the technical solutions in this invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are some embodiments of this invention. For those skilled in the art, other drawings can be obtained from these drawings without creative effort.

[0063] Figure 1 This is a flowchart of a machine vision-based car seat posture recognition method provided in an embodiment of the present invention;

[0064] Figure 2 This is a flowchart illustrating the determination of the danger level of a current seating posture in a machine vision-based car seat posture recognition method provided in an embodiment of the present invention.

[0065] Figure 3 This is a flowchart illustrating the generation of corresponding safety response decisions in a machine vision-based car seat posture recognition method provided in an embodiment of the present invention.

[0066] Figure 4 This is a schematic diagram of a machine vision-based car seat posture recognition system provided in an embodiment of the present invention. Detailed Implementation

[0067] To make the objectives, technical solutions, and advantages of this invention clearer, the technical solutions of this invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some, not all, of the embodiments of this invention. All other embodiments obtained by those skilled in the art based on the embodiments of this invention without creative effort are within the scope of protection of this invention.

[0068] Example 1:

[0069] The following is combined with Figures 1-4This invention describes a machine vision-based method and system for recognizing the posture of a car seat.

[0070] like Figure 1 As shown in the figure, an embodiment of the present invention provides a method for recognizing the posture of a car seat based on machine vision, comprising:

[0071] S1: Real-time acquisition of pressure data and pressure distribution characteristics; by setting pressure thresholds and combining pressure distribution characteristics, it determines the presence of seat occupants and outputs the occupant assessment results.

[0072] The steps for determining the presence of seat occupants include:

[0073] S11: A pressure sensor array deployed on the seat surface collects pressure values ​​from each sensor in real time. The pressure sensor array is an 8×8 distributed piezoresistive sensor array with a sensor spacing of 5cm×5cm. The deployment area covers the front 2 / 3 of the seat cushion and the lower middle part of the backrest. The sampling frequency is set to 10Hz. Each sensor outputs a 0-5V analog voltage signal, which is converted into a digital quantity P of 0-65535 by a 16-bit AD converter. i (i=1,2,...,64, representing sensor numbers).

[0074] S12: Extract pressure distribution features based on pressure values. Pressure distribution features include the pressure coverage area S (number of effective pressure sensors). ), pressure concentration index C (the sum of the pressure values ​​of the top 20% of sensors / total pressure value) and pressure center offset D (the Euclidean distance between the pressure center and the geometric center of the seat).

[0075] S13: Set pressure threshold conditions and determine the presence of seat occupants based on the pressure values, then output the occupant determination result. Pressure threshold conditions include a static threshold P0 and a dynamic threshold P. dyn =α×P avg Where α is the environmental correction factor (corrected by ±5% for every 10℃ change in temperature), P avg This represents the average background pressure value over the past three cycles. Judgment rule: When three or more sensor pressure values ​​are greater than P0 and the total pressure value is greater than P within three consecutive cycles... dyn When the pressure value of all sensors is less than P0 / 2 for 5 consecutive cycles, it is determined to be "unoccupied"; when the pressure value of all sensors is less than P0 / 2 for 5 consecutive cycles, it is determined to be "unoccupied". The static threshold P0 is set to 10% of the full scale of the sensor, that is, based on the full scale of the 8×8 sensor array. It is determined by statistically recording the maximum pressure fluctuation under no-load conditions through 100 sets of no-load and manned experiments.

[0076] S2: Use the occupant assessment result as a trigger signal to acquire occupant image data.

[0077] The steps for acquiring occupant image data include:

[0078] S21: When the occupant assessment result indicates that there are occupants, the in-vehicle image acquisition device is triggered.

[0079] S22: Synchronously acquire RGB and depth images of the occupant's upper body, with the acquisition frequency consistent with the acquired pressure data. Timestamp alignment is achieved via a hardware synchronization trigger signal, with a synchronization error ≤1ms. Each acquisition cycle generates a pair of RGB and depth images with the same timestamp.

[0080] S23: If the occupant assessment result is no occupants, the image acquisition device enters sleep mode. In sleep mode, the device power consumption drops to less than 5% of the normal operating state, and it wakes up every 30 seconds to perform a 100ms status check.

[0081] S3: Preprocess and extract features from the occupant image data to obtain image feature data. Extract features from the pressure data and pressure distribution characteristics to obtain pressure feature data.

[0082] Step S3, which involves preprocessing and extracting features from the occupant image data, includes:

[0083] S31: Preprocess the RGB and depth images, and output the preprocessed image. Preprocessing includes: denoising the RGB image using a 3×3 Gaussian filter (Gaussian kernel equal to 1.0), and enhancing contrast using the CLAHE algorithm. For the depth image, median filtering is used to remove salt-and-pepper noise, bilateral filtering is used to preserve edge information, and neighborhood interpolation is used to fill invalid depth values ​​(e.g., >3m or <0.5m).

[0084] S32: Based on the preprocessed image, a human pose estimation algorithm is used to extract the coordinates of key skeletal points, limb contour features, and relative positional relationships of body parts of the occupant, outputting preliminary image feature data. An improved OpenPose algorithm is used to extract 18 key skeletal points, which are then labeled. Improvements include: adding adaptive histogram equalization preprocessing to address uneven lighting inside the vehicle; using neighborhood skeletal point motion trend prediction to complete occluded skeletal points (such as the wrist when the arm is resting on the ground); and optimizing the keypoint confidence threshold (increasing it from the default 0.5 to 0.65) to reduce false detections.

[0085] The coordinates of the skeletal points are denoted by the top-left corner of the image as the origin, and are denoted as (x... k ,y k ), k=1,2,...,18. Key skeletal points are usually the head, neck, left shoulder, right shoulder, left elbow, right elbow, left wrist, right wrist, left hip, right hip, left scapula, right scapula, left waist, right waist, etc.

[0086] S33: Perform 3D information analysis on the depth image, and combine it with preliminary image feature data to obtain the spatial depth coordinates and limb bending angle features of various parts of the occupant's upper body, thus obtaining image feature data. The 3D coordinate calculation method is to calculate the pixel coordinates (x, y, x) of key skeletal points of the occupant's upper body in the RGB image or depth image based on the depth camera intrinsic parameter matrix K. k ,y k ) and depth value d k Convert to three-dimensional spatial coordinates (X k ,Y k Z k ), represented as:

[0087]

[0088]

[0089]

[0090]

[0091] Among them, f x f is the focal length of a camera in the horizontal direction, usually measured in pixels. It reflects the camera's optical system's ability to converge light and determines the mapping relationship between pixels and actual spatial distances in the horizontal direction. y The focal length of the camera in the vertical direction, usually expressed in pixels, is related to f / 2. x Similarly, it's used for vertical mapping. `cx` represents the principal coordinates of the image plane in the horizontal direction, i.e., the x-coordinate of the intersection of the optical axis and the image plane, in pixels, and is the origin offset of the pixel coordinate system. `cy` represents the principal coordinates of the image plane in the vertical direction, i.e., the y-coordinate of the intersection of the optical axis and the image plane, in pixels, and is used for the origin offset in the vertical direction.

[0092] The limb flexion angle is calculated using the vector cosine theorem. For example, the elbow joint angle is calculated as follows:

[0093]

[0094]

[0095]

[0096] Here, θ5 represents the limb bending angle, taking the elbow joint as an example. It is calculated by combining the dot product operation between vectors with the law of cosines, reflecting the degree of limb bending. 35 m 57These are the shoulder-elbow vector and the elbow-wrist vector, respectively. They are vectors calculated from the spatial coordinates of the corresponding skeletal points and are used to solve for the limb bending angle through vector operations. (X3,Y3,Z3) are the spatial coordinates of the left shoulder skeletal point, (X5,Y5,Z5) are the spatial coordinates of the left elbow skeletal point, and (X7,Y7,Z7) are the three-dimensional spatial coordinates of the occupant's left wrist skeletal point in the camera coordinate system.

[0097] Step S3, which involves feature extraction from the pressure data and pressure distribution characteristics, includes:

[0098] S34: Based on the pressure value, calculate the pressure center coordinates, pressure distribution entropy, and pressure gradient change rate, and output the overall pressure data.

[0099] In step S34, the calculation steps for the overall pressure data include:

[0100] S341: Calculate the pressure center coordinates using a weighted average based on the pressure values ​​and corresponding coordinates. The lower left corner of the seat cushion is designated as the origin. Pressure center coordinates (x... c ,y c Calculation formula:

[0101]

[0102] Among them, (x i ,y i Let ΣP be the physical coordinates of the i-th sensor. i This is the sum of the pressure values ​​from all sensors.

[0103] S342: Based on the pressure values, calculate the proportion of each sensor's pressure value to the total pressure value, and calculate the pressure distribution entropy using information entropy. The formula for calculating the pressure distribution entropy H is expressed as:

[0104]

[0105] Where, p i For the pressure percentage of the i-th sensor, when p i p is defined when =0 i ·log2p i =0, and the range of H is (0, log264), which is (0, 6).

[0106] S343: Based on the pressure values ​​of each sensor and the seat's preset zones, calculate the total pressure of all sensors within each zone, determine the percentage of pressure in each zone relative to the total pressure, and output the pressure percentage for each area. The seat is preset into four zones: Left Front Zone A (sensors 1-16), Right Front Zone B (sensors 17-32), Left Rear Zone C (sensors 33-48), and Right Rear Zone D (sensors 49-64). The calculation method for the area pressure percentage is as follows:

[0107]

[0108]

[0109] in, This represents the sum of the sensor pressures in area A. This represents the sum of the pressures from the sensors in zone B. This represents the sum of the sensor pressures in zone C. This represents the total pressure from the sensors in zone D.

[0110] S344: Calculate the pressure gradient change rate G based on pressure gradient data from multiple acquisition cycles. The formula for calculating the pressure gradient change rate G is as follows:

[0111]

[0112] Where t is the current sampling period, The total pressure of the current cycle, This represents the total pressure from the previous cycle. This is the sampling interval.

[0113] S35: The pressure values ​​of the pressure sensor array are divided into regions using a clustering algorithm. The peak pressure, pressure duration, and pressure change frequency of each region are extracted, and the regional pressure data is output. The K-means clustering algorithm (K=3-5, optimal K is determined using the elbow method, and then verified by the profile coefficient; a profile coefficient >0.6 is considered reasonable) is used to group sensors with similar pressure values ​​into the same region. Regional pressure peak P peak The maximum pressure value within the region; the pressure duration T is the region's pressure value > P. peak ×50% of the number of consecutive cycles ×0.1s; the pressure change frequency F is calculated by Fourier transform, and the main frequency in the 0.5-5Hz frequency band is taken.

[0114] S36: Normalize and fuse the overall pressure data and regional pressure data to obtain pressure characteristic data. Normalization uses min-max standardization, expressed as:

[0115]

[0116] Where b' is the normalized eigenvalue, b is the original eigenvalue, and b min b max The features are the historical minimum and maximum values. Fusion is achieved through feature concatenation; the pressure feature data dimensions are overall pressure data (5 dimensions) + regional pressure data (3×K dimensions).

[0117] S4: Analyze the occupant type based on image feature data and pressure feature data, and output the occupant type analysis results.

[0118] Step S4, the steps for analyzing occupant type, include:

[0119] S41: Based on image feature data, determine the occupant's height range and body shape characteristics. The height range L is calculated using the spatial distance between the head and the hip bone points.

[0120]

[0121] Where Z1 is the Z-coordinate of the head, Z9 is the Z-coordinate of the left hip, and the error correction coefficient is 1.2. Body shape characteristics are calculated using the shoulder width / hip width ratio R, and the calculation method is as follows:

[0122]

[0123] Where X3 represents the X-axis spatial coordinates of the occupant's left shoulder key skeletal point in the camera coordinate system, indicating the horizontal position of the left shoulder in the camera coordinate system. X4 represents the X-axis spatial coordinates of the occupant's right shoulder key skeletal point in the camera coordinate system, reflecting the horizontal position of the right shoulder in the camera coordinate system. X9 represents the X-axis spatial coordinates of the occupant's left hip key skeletal point in the camera coordinate system, indicating the horizontal position of the left hip in the camera coordinate system. 10 R represents the X-axis spatial coordinates of the key bone point of the occupant's right hip in the camera coordinate system, indicating the horizontal position of the right hip in the camera coordinate system. R>1.1 indicates a slender type, 0.9≤R≤1.1 indicates a symmetrical type, and R<0.9 indicates a robust type.

[0124] S42: Based on pressure characteristic data, determine the occupant's weight range. The formula for estimating weight M is:

[0125]

[0126] in, The average total pressure over 5 cycles is represented by k, a conversion factor, and b, a correction value typically set to 3.5 kg. This value is obtained by fitting data from 100 sets of samples. The weight ranges are divided into: <50 kg, 50-65 kg, 65-80 kg, and >80 kg.

[0127] S43: Input the height range, body shape characteristics, and weight range into the pre-trained classification model, and output the passenger type analysis results. The classification model adopts a random forest model. The input features are height range (discrete into 4 categories), body shape characteristics (3 categories), and weight range (4 categories). The output passenger types include: children (L<120cm), adult women (L in the range of 120-175cm, body shape coefficient adjusted), adult men (L in the range of 150-190cm, body shape coefficient adjusted), obese adults (M>80kg and R<0.85), and unknown (placed objects, pets, etc.).

[0128] S5: The image feature data and pressure feature data are fused, and the fused feature data is output. This fused feature data is then input into a pre-trained posture recognition model. Combined with the occupant type analysis results, the occupant's posture is recognized, and the posture recognition result is output. The feature fusion uses weighted feature fusion, as shown below:

[0129]

[0130] in, To fuse feature vectors, F img F is the image feature vector. pressure The pressure feature vector is represented by ω1=0.6 and ω2=0.4, which are the weights. The sitting posture recognition model uses a CNN-LSTM model, which inputs fused features and outputs sitting posture types including: normal sitting posture, forward-leaning sitting posture, side-leaning sitting posture, excessive backward-leaning posture, legs crossed, and arms draped over the arm, totaling 8 categories.

[0131] The steps for building a CNN-LSTM model include:

[0132] A large amount of image and pressure feature data was collected from different occupants (covering different height ranges, body shapes, and weight ranges) in various sitting postures. The image data includes RGB images and depth images, and the pressure data includes various calculated features such as pressure center coordinates and pressure distribution entropy. Simultaneously, the correct sitting posture type was labeled for each data set.

[0133] The collected data is divided into training, validation, and test sets. Typically, the training set accounts for 60%-80%, the validation set for 10%-20%, and the test set for 10%-20%. For example, 80% is used as the training set to train model parameters, 10% as the validation set to tune model hyperparameters, and 10% as the test set to evaluate the generalization performance of the final model.

[0134] The RGB image is normalized, scaling pixel values ​​to the [0,1] or [-1,1] range. A similar normalization operation is performed on the depth image, ensuring the correct correspondence between skeletal point coordinates and other data and the image data. A min-max normalization method is used to map pressure-related feature values ​​to the [0,1] range. A weighted feature fusion method is then used to fuse the preprocessed image feature vector and the pressure feature vector to obtain the final feature data for the input model.

[0135] The part about building a CNN (Convolutional Neural Network).

[0136] Define an input layer, the dimension of which is determined by the dimension of the fused features, to receive the preprocessed fused feature data. Add convolutional layers, typically multiple layers stacked. The number of kernels in subsequent convolutional layers can be increased appropriately. Add pooling layers after the convolutional layers; pooling reduces data dimensionality, decreases computation, and can also prevent overfitting to some extent. After convolution and pooling operations, the data is flattened and then connected to fully connected layers.

[0137] The part about building an LSTM (Long Short-Term Memory) network.

[0138] An LSTM layer is added after the fully connected layer. LSTM can process data with time-series characteristics and has a good ability to capture continuous posture changes that may exist in posture recognition. Finally, a fully connected layer is added as the output layer to output the predicted posture type.

[0139] For model optimizers, Adam is commonly used. The Adam optimizer can adaptively adjust the learning rate, leading to faster convergence during training. The cross-entropy loss function is typically chosen to measure the difference between the predicted probability distribution and the true label distribution. Finally, accuracy is selected as the evaluation metric.

[0140] During model training, the number of training epochs and batch size need to be determined. The model is trained using the training set data by calling the `model.fit` function, while parameters are adjusted using validation set data to prevent overfitting. During model evaluation, the `model.evaluate` function is called to obtain the test loss and accuracy on the test set. If the accuracy is low, the model is optimized by adjusting the model structure, preprocessing methods, or hyperparameters, and then retrained and evaluated. During model deployment, the high-performing model is saved as a specific file and integrated into the car seat posture recognition system, interfacing with the pressure and image acquisition modules to achieve real-time posture recognition. The `model.fit` function is the core function for training the model. It iteratively optimizes the model parameters, enabling the model to learn the mapping relationship between input data and labels. Through the iterative process, `model.fit` gradually optimizes the model from randomly initialized parameters, ultimately enabling it to accurately predict the posture type based on the input posture features—a crucial step in the formation of the model's "learning ability." The `model.evaluate` function is a Keras function used to evaluate the performance of a trained model. When `model.evaluate` is called, the model sequentially inputs data samples from `x_test` into the model for forward propagation to obtain prediction results. Then, based on the prediction results and the true labels in y_test, the test loss is calculated according to the loss function set during model compilation. Simultaneously, the number of correctly predicted samples is counted, and the test accuracy is calculated.

[0141] S6: Combining the occupant type analysis results and the sitting posture recognition results, the risk level of the current sitting posture is determined by comparing and analyzing the risk level of different sitting postures under the corresponding occupant type according to the risk level assessment criteria, and the risk analysis results are output.

[0142] like Figure 2 As shown, step S6, determining the hazard level of the current riding posture, includes:

[0143] S61: Preset basic hazard level threshold matrix. The elements in the basic hazard level threshold matrix are the hazard values ​​corresponding to different combinations of occupant types and sitting postures. Basic hazard threshold matrix D base The matrix is ​​5×8 (5 types of occupants × 8 types of sitting postures), with a risk value range of 1-10. For example, adult males + normal sitting posture = 2, children + forward-leaning sitting posture = 8.

[0144] S62: Real-time acquisition of vehicle driving status data and seat belt status signals. Vehicle driving status data includes vehicle speed v, acceleration a, and steering angle γ; seat belt status signal S... belt It is in binary (1 = worn, 0 = not worn).

[0145] S63: Based on vehicle driving status data and seat belt status signals, a fuzzy logic controller dynamically adjusts the basic hazard level threshold by inputting vehicle dynamic parameters and occupant posture parameters into a membership function. The fuzzy logic controller input variables include: vehicle speed membership function μ. v (v) = min(1, v / 60), acceleration membership function μ a (a) = min(1, |a| / 5), seat belt correction factor k belt =1.5 (not worn) or k belt =1.0 (wearing). The fuzzy rule base can specifically include: if the vehicle speed is high (μ... v >0.8) and not wearing a seatbelt (k belt =1.5), then the danger value correction factor = 1.2; if the acceleration is large (μ a If the risk level is >0.7 and the sitting posture is forward-leaning, the correction factor is 1.3; otherwise, the correction factor is 1.0. The corrected risk value is expressed as follows:

[0146]

[0147] Where D is the corrected danger value, D base Based on the hazard threshold matrix.

[0148] S64: When an emergency situation is detected, the danger level escalation mechanism is automatically triggered, forcibly increasing the danger level of the current attitude. Emergency situation determination: Rapid acceleration. Emergency braking Sharp turn Danger value after triggering .

[0149] S7: Based on the hazard analysis results, generate corresponding safety response decisions.

[0150] like Figure 3 As shown, step S7, the steps for generating the corresponding security response decision, include:

[0151] S71: Based on the results of hazard analysis, three levels of response are defined, including: Level 1 (D≤3), Level 2 (4≤D≤6), and Level 3 (D≥7).

[0152] S72: In Level 1 response, a posture correction prompt is displayed on the vehicle's central control screen, accompanied by a gentle prompt tone. The prompt tone frequency is 1kHz, lasts for 0.5s, and repeats at 5s intervals. The prompt text reads, "Please maintain correct posture for safety."

[0153] S73: In Level 2 response, in addition to Level 1 response measures, seat vibration feedback is activated, and current posture data is recorded simultaneously. The seat vibration uses a 3Hz low-frequency vibration, lasting for 2 seconds, with the vibration intensity increasing linearly with the danger level; the recorded data includes image frames, pressure characteristics, and vehicle status, and is stored for 30 minutes.

[0154] S74: At Level 3 response, an active intervention mechanism is triggered. This mechanism includes: if the vehicle is traveling at low speed, an automatic deceleration warning and suggestion to stop and adjust the seat are issued. If traveling at high speed, with occupant confirmation, a minor posture correction is performed via the electric seat adjustment device, and a warning signal is simultaneously sent to the driver's seat. Specifically, a "Adjust seat?" dialog box pops up on the central control screen (default rejection if no action is taken within 3 seconds), along with a voice prompt "Please confirm seat adjustment," and minor correction is performed upon confirmation. The minor correction amount Δ = β × (D - 6), where β is the adjustment coefficient (seat cushion angle 0.5° / danger value, backrest angle 0.3° / danger value), with a maximum adjustment amount ≤ 3°. The driver's seat warning is achieved through flashing instrument panel icons and a buzzer.

[0155] S75: Based on the occupant's response to corrective cues, reinforcement learning is used to optimize the trigger thresholds and execution methods of subsequent response strategies. The reinforcement learning state includes the current danger level, response type, and occupant feedback (corrected / uncorrected). The reward function is: +10 for correcting posture, -5 for not correcting, and an additional +15 for three consecutive corrective actions. The strategy parameters are updated every 100 interactions.

[0156] In summary, the present invention provides a machine vision-based method and system for recognizing the posture of car seats. Through multi-dimensional fusion perception of pressure data and image data, it achieves accurate judgment and dynamic monitoring of the presence of seat occupants, effectively reducing energy consumption from invalid image acquisition and improving system operating efficiency. By comprehensively judging occupant type based on height, body shape, and weight, posture recognition becomes more targeted. By employing a three-level response mechanism and active intervention strategy, the warning and correction methods can be dynamically adjusted according to the level of danger. Combined with reinforcement learning for continuous optimization of the strategy, this significantly improves vehicle passenger safety, providing personalized and intelligent safety protection for occupants and effectively reducing driving risks caused by poor posture.

[0157] like Figure 4 As shown, the present invention also provides a car seat posture recognition system based on machine vision, including: a pressure sensor array, a pressure analysis module, an image acquisition module, an image processing module, an occupant type analysis module, a posture recognition module, a hazard level classification module, and a safety response decision generation module.

[0158] Pressure sensor arrays are used to acquire pressure data in real time.

[0159] The pressure analysis module is used to obtain pressure distribution characteristics based on pressure data, determine the presence of seat occupants by setting pressure thresholds, and output the occupant assessment results. It also extracts features from the pressure data and pressure distribution characteristics to obtain pressure feature data.

[0160] The image acquisition module is used to acquire occupant image data based on the occupant assessment results.

[0161] The image processing module is used to preprocess and extract features from occupant image data to obtain image feature data.

[0162] The occupant type analysis module is used to analyze occupant type based on image feature data and pressure feature data, and output the occupant type analysis results.

[0163] The posture recognition module fuses image feature data and pressure feature data, outputting the fused feature data. This fused feature data is then input into a pre-trained posture recognition model, which, combined with occupant type analysis results, performs occupant posture recognition and outputs the posture recognition result.

[0164] The hazard level classification module combines the occupant type analysis results and the sitting posture recognition results, compares and analyzes the hazard level of different sitting postures under the corresponding occupant type according to the hazard level judgment criteria, determines the hazard level of the current sitting posture, and outputs the hazard analysis results.

[0165] The safety response decision generation module is used to generate corresponding safety response decisions based on the hazard analysis results.

[0166] Through the above description of the embodiments, those skilled in the art can clearly understand that each embodiment can be implemented by means of software plus necessary general-purpose hardware platforms, and of course, it can also be implemented by hardware. Based on this understanding, the above technical solutions, in essence or the part that contributes to the prior art, can be embodied in the form of a software product. This computer software product can be stored in a computer-readable storage medium, such as ROM / RAM, magnetic disk, optical disk, etc., and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute the methods described in the various embodiments or some parts of the embodiments.

[0167] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, and not to limit them; although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features; and these modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. A machine vision-based method for recognizing the posture of a car seat, characterized in that, include: S1: Real-time acquisition of pressure data and pressure distribution characteristics; by setting a pressure threshold and combining the pressure distribution characteristics, determine the presence of seat occupants and output the occupant determination result; S2: Use the occupant determination result as a trigger signal to collect occupant image data; S3: Preprocess and extract features from the occupant image data to obtain image feature data; and extract features from the pressure data and pressure distribution characteristics to obtain pressure feature data; S4: Analyze the occupant type based on the image feature data and the pressure feature data, and output the occupant type analysis results; S5: Perform feature fusion on the image feature data and the pressure feature data, and output the fused feature data; input the fused feature data into the pre-trained sitting posture recognition model, combine it with the occupant type analysis results, perform occupant sitting posture recognition, and output the sitting posture recognition result; S6: Combining the occupant type analysis results and the sitting posture recognition results, compare and analyze the different sitting postures according to the risk level assessment criteria for the corresponding occupant type, determine the risk level of the current sitting posture, and output the risk analysis results; Step S6, determining the hazard level of the current riding posture, includes: S61: A preset basic hazard level threshold matrix is ​​defined, wherein the elements in the basic hazard level threshold matrix are the hazard values ​​corresponding to combinations of different occupant types and sitting postures. S62: Real-time acquisition of vehicle driving status data and seat belt status signals; S63: Based on the vehicle driving status data and seat belt status signal, the vehicle dynamic parameters and occupant posture parameters are input into the membership function through a fuzzy logic controller to dynamically correct the basic hazard level threshold. S64: When an emergency situation is detected, the danger level jump mechanism is automatically triggered, forcibly increasing the danger level of the current posture; S7: Based on the hazard analysis results, generate corresponding safety response decisions, including the following steps: S71: Based on the results of hazard analysis, response levels are divided into three levels: Level 1, Level 2, and Level 3; S72: In Level 1 response, a posture correction prompt is displayed on the in-vehicle central control screen, accompanied by a gentle prompt tone; S73: In the second-level response, in addition to the first-level response measures, activate the seat vibration feedback and record the current posture data; S74: In the third-level response, an active intervention mechanism is triggered; the active intervention mechanism includes: if the vehicle is traveling at low speed, automatically issuing a deceleration reminder and suggesting stopping to adjust; if the vehicle is traveling at high speed, with the occupant's confirmation, performing slight posture correction through the seat electric adjustment device and simultaneously sending a warning signal to the driver's seat. S75: Based on the occupant's response to corrective cues, optimize the trigger threshold and execution method of subsequent coping strategies through reinforcement learning.

2. The method for recognizing car seat posture based on machine vision according to claim 1, characterized in that, Step S1, the steps for determining the presence of seat occupants, include: S11: Real-time acquisition of pressure values ​​from each sensor via an array of pressure sensors deployed on the seat surface; S12: Extract the pressure distribution features based on the pressure value; S13: Set pressure threshold conditions, and determine the presence of seat occupants based on the pressure value, and output the occupant determination result.

3. The method for recognizing car seat posture based on machine vision according to claim 1, characterized in that, Step S2, the steps for acquiring occupant image data, include: S21: When the occupant determination result indicates that there are occupants, the in-vehicle image acquisition device is triggered; S22: Simultaneously acquire RGB and depth images of the upper body of the occupant, with the acquisition frequency consistent with the acquisition pressure data; S23: If the occupant determination result is no occupants, the image acquisition device is in a sleep state.

4. The method for recognizing car seat posture based on machine vision according to claim 3, characterized in that, Step S3, the steps of preprocessing and feature extraction of the occupant image data, include: S31: Preprocess the RGB image and depth image, and output the preprocessed image; S32: Based on the preprocessed image, a human pose estimation algorithm is used to extract the coordinates of key skeletal points, limb contour features and relative positional relationships of body parts of the occupant, and output preliminary image feature data. S33: Perform three-dimensional information analysis on the depth image, and combine it with the preliminary image feature data to obtain the spatial depth coordinates and limb bending angle features of each part of the upper body of the occupant, thereby obtaining the image feature data.

5. The method for recognizing car seat posture based on machine vision according to claim 2, characterized in that, Step S3, the step of feature extraction of the pressure data and pressure distribution features, includes: S34: Based on the pressure value, calculate the pressure center coordinates, pressure distribution entropy, and pressure gradient change rate, and output the overall pressure data; S35: The pressure values ​​of the pressure sensor array are divided into regions using a clustering algorithm. The pressure peak value, pressure duration and pressure change frequency of each region are extracted, and the regional pressure data is output. S36: Normalize the overall pressure data and the regional pressure data, and then fuse them to obtain the pressure characteristic data.

6. The method for recognizing car seat posture based on machine vision according to claim 5, characterized in that, In step S34, the calculation steps for the overall pressure data include: S341: Calculate the pressure center coordinates by weighted average based on the pressure value and corresponding coordinates; S342: Based on the pressure value, calculate the proportion of each sensor pressure value to the total pressure value, and calculate the pressure distribution entropy using the information entropy method; S343: Based on the pressure values ​​of each sensor and the preset zones of the seat, calculate the total pressure of all sensors in each zone, calculate the percentage of pressure in each zone to the total pressure, and output the pressure percentage of each zone. S344: Calculate the rate of change of pressure gradient based on pressure gradient data from multiple acquisition cycles.

7. The method for recognizing car seat posture based on machine vision according to claim 4, characterized in that, Step S4, the steps for analyzing occupant type, include: S41: Based on the image feature data, determine the height range and body shape characteristics of the occupants; S42: Based on the pressure characteristic data, determine the occupant's weight range; S43: Input the height range, body shape characteristics and weight interval into the pre-trained classification model, and output the passenger type analysis results.

8. A machine vision-based automotive seat posture recognition system, which employs a machine vision-based automotive seat posture recognition method as described in any one of claims 1 to 7, characterized in that, include: Pressure sensor array for real-time pressure data acquisition; The pressure analysis module is used to obtain pressure distribution characteristics based on the pressure data, determine the presence of seat occupants by setting a pressure threshold, and output the occupant determination result; and to extract features from the pressure data and pressure distribution characteristics to obtain pressure feature data. The image acquisition module is used to acquire occupant image data based on the occupant judgment result; The image processing module is used to preprocess and extract features from the occupant image data to obtain image feature data; The occupant type analysis module is used to analyze the occupant type based on the image feature data and the pressure feature data, and output the occupant type analysis results. The posture recognition module is used to fuse the image feature data and the pressure feature data, and output the fused feature data; input the fused feature data into the pre-trained posture recognition model, and combine it with the occupant type analysis results to perform occupant posture recognition and output the posture recognition result. The hazard level classification module is used to combine the occupant type analysis results and the sitting posture recognition results, compare and analyze the hazard level of different sitting postures under the corresponding occupant type according to the hazard level judgment criteria, determine the hazard level of the current sitting posture, and output the hazard analysis results. The safety response decision generation module is used to generate corresponding safety response decisions based on the hazard analysis results.

Images