Display height self-adaptive adjusting method and system based on user posture recognition

Abstract

The invention discloses a display height adaptive adjustment method and system based on user posture recognition, and the method comprises the steps: processing a user posture image through a lightweight MTCNN model, dynamically adjusting the image contrast based on a Retinex theory, and obtaining an enhanced user posture image; detecting eye pixel coordinates and shoulder key point coordinates in the enhanced user posture image by using a lightweight HRNet model, and converting the eye pixel coordinates into three-dimensional space coordinates with the center of a display as an original point in combination with depth map auxiliary distance data; acquiring a comfortable visual angle established by initial calibration of the user, calculating a sitting posture compensation amount based on the shoulder key point coordinates, and calculating a target height according to the sitting posture compensation amount; and performing Kalman filtering on the three-dimensional space coordinates, predicting an attitude change trend through a lightweight LSTM network, and generating an adaptive adjustment instruction according to the attitude change trend and the target height. And the height self-adaptive adjustment efficiency and accuracy of the display are improved.

Description

Technical Field

[0001] This invention relates to the field of human-computer interaction technology, and in particular to a method and system for adaptive adjustment of display height based on user posture recognition. Background Technology

[0002] Monitor height adaptability directly impacts user visual comfort and physical health. Current monitor height adjustment methods mostly rely on manual adjustment, depending on user judgment. This results in low precision, cumbersome operation, and an inability to adapt to dynamic changes in user posture. Some automatic adjustment solutions rely solely on data from a single distance sensor, failing to consider user posture characteristics, leading to poor accuracy and difficulty in matching the comfort viewing angle needs of different users. Furthermore, changes in ambient lighting can easily blur user posture recognition, resulting in insufficient accuracy in key point detection and consequently, low efficiency and accuracy of adaptive monitor height adjustment. Summary of the Invention

[0003] The purpose of this invention is to solve the above problems by designing a display height adaptive adjustment method and system based on user posture recognition.

[0004] To achieve the above objectives, the technical solution of the present invention further includes the following steps in the above-mentioned display height adaptive adjustment method based on user posture recognition:

[0005] The user's posture image and depth map are simultaneously acquired by the camera and ToF sensor to assist in distance data acquisition.

[0006] The user pose image is processed by a lightweight MTCNN model, and the image contrast is dynamically adjusted based on Retinex theory to obtain an enhanced user pose image.

[0007] The lightweight HRNet model is used to detect the eye pixel coordinates and shoulder key point coordinates in the enhanced user pose image. The eye pixel coordinates are converted into three-dimensional spatial coordinates with the center of the display as the origin by combining depth map auxiliary distance data.

[0008] Obtain the comfortable viewing angle established by the user's initial calibration, calculate the sitting posture compensation amount based on the shoulder key point coordinates, and calculate the target height based on the sitting posture compensation amount;

[0009] Kalman filtering is applied to the three-dimensional spatial coordinates, and the attitude change trend is predicted through a lightweight LSTM network. Adaptive adjustment commands are then generated based on the attitude change trend and the target height.

[0010] Furthermore, in the aforementioned display height adaptive adjustment method based on user pose recognition, the step of processing the user pose image using a lightweight MTCNN model and dynamically adjusting the image contrast based on Retinex theory to obtain an enhanced user pose image includes:

[0011] Based on the pre-trained MTCNN model, the MTCNN model is pruned and optimized by removing redundant convolutional layers and retaining the three-layer structure of P-Net, R-Net and O-Net. The model input is the collected user pose image and the output is the filtered candidate boxes of facial regions.

[0012] The multi-scale Retinex algorithm in Retinex theory is used to perform illumination compensation and contrast enhancement on candidate bounding boxes of facial regions. The image is converted into a grayscale image and decomposed into reflection and illumination components. The illumination component is extracted by Gaussian filtering, and the reflection component is enhanced by adaptive gain adjustment. The enhanced user pose image is then reconstructed.

[0013] Furthermore, in the aforementioned display height adaptive adjustment method based on user pose recognition, the step of using a lightweight HRNet model to detect the eye pixel coordinates and shoulder key point coordinates in the enhanced user pose image, and combining depth map-assisted distance data to convert the eye pixel coordinates into three-dimensional spatial coordinates with the display center as the origin, includes:

[0014] Based on the pre-trained lightweight HRNet model, a 1×1 convolutional kernel is used to replace the 3×3 convolutional kernel, and the number of channels is compressed to half of the original model, resulting in an optimized lightweight HRNet model.

[0015] The enhanced user pose image is input into the optimized lightweight HRNet model, which outputs eye pixel coordinates and shoulder keypoint coordinates, as well as the confidence score of the keypoints. Keypoints with a confidence score below 0.9 are discarded and filled in by interpolation from adjacent frames.

[0016] Furthermore, in the aforementioned display height adaptive adjustment method based on user pose recognition, the step of using a lightweight HRNet model to detect the eye pixel coordinates and shoulder key point coordinates in the enhanced user pose image, and combining depth map-assisted distance data to convert the eye pixel coordinates into three-dimensional spatial coordinates with the display center as the origin, includes:

[0017] The eye pixel coordinates and shoulder keypoint coordinates output by the HRNet model are filtered to remove coordinate points with insufficient confidence or abnormal positions; median filtering is used to smooth the coordinate points of consecutive frames to eliminate keypoint jitter caused by random noise.

[0018] The average coordinates of the centers of the left and right pupils are calculated as the final eye pixel coordinates, and the average coordinates of the vertices of the left and right shoulders are calculated as the shoulder keypoint coordinates.

[0019] Extract the depth values ​​corresponding to the eye pixel coordinates from the auxiliary distance data of the depth map; establish a three-dimensional spatial coordinate system with the center of the display as the origin, defining the X-axis as the horizontal direction of the display, the Y-axis as the vertical direction of the display, and the Z-axis as the direction perpendicular to the display screen;

[0020] Based on the camera's intrinsic parameters and depth values, the two-dimensional pixel coordinates of the eye are converted into three-dimensional spatial coordinates using a formula that converts pixel coordinates to three-dimensional coordinates, and the three-dimensional spatial coordinates of the eye are output.

[0021] Furthermore, in the above-mentioned display height adaptive adjustment method based on user posture recognition, the steps of obtaining the comfortable viewing angle established by the user's initial calibration, calculating the sitting posture compensation amount based on the shoulder key point coordinates, and calculating the target height based on the sitting posture compensation amount include:

[0022] Guide users to perform comfortable viewing angle calibration, collect real-time three-dimensional spatial coordinates of the eyes and real-time key point coordinates of the shoulders, record the initial display height, calculate the vertical angle between the eyes and the center of the display, and set the vertical angle range of the comfortable viewing angle to ±3°.

[0023] The vertical sitting posture offset is calculated by comparing the real-time shoulder key point coordinates with the actual shoulder key point coordinates, and the sitting posture compensation is calculated based on the sitting posture offset.

[0024] Furthermore, in the above-mentioned display height adaptive adjustment method based on user posture recognition, the steps of obtaining the comfortable viewing angle established by the user's initial calibration, calculating the sitting posture compensation amount based on the shoulder key point coordinates, and calculating the target height based on the sitting posture compensation amount include:

[0025] Based on the real-time three-dimensional spatial coordinates of the eyes, calculate the vertical angle between the current eyes and the center of the monitor. Compare the current vertical angle with the vertical angle of the comfortable viewing angle to calculate the angle deviation. Based on the angle deviation, the initial monitor height, and the posture compensation amount, calculate the target height of the monitor.

[0026] The target height is constrained; if it exceeds the range, the corresponding boundary value is taken as the final target height.

[0027] Furthermore, in the above-mentioned display height adaptive adjustment method based on user pose recognition, the step of performing Kalman filtering on the three-dimensional spatial coordinates, predicting the pose change trend through a lightweight LSTM network, and generating adaptive adjustment instructions based on the pose change trend and the target height includes:

[0028] A lightweight LSTM network is constructed with an input layer dimension of 3, 16 hidden units in the LSTM hidden layer, and a dropout probability of 0.2 in the dropout layer to prevent overfitting. The output layer dimension of the fully connected layer is 3, corresponding to the predicted value of the three-dimensional coordinates of the eye.

[0029] The predicted value is compared with the filtered three-dimensional spatial coordinates to calculate the attitude change. Based on the sign and absolute value of the attitude change, the trend of the user's attitude change is determined. An adaptive adjustment command is generated based on the attitude change trend and the target height.

[0030] Furthermore, in the display height adaptive adjustment system based on user posture recognition, the display height adaptive adjustment system includes the following modules:

[0031] The posture image acquisition module is used to simultaneously acquire user posture images and depth maps to assist distance data through a camera and a ToF sensor;

[0032] The image data enhancement module is used to process the user pose image using a lightweight MTCNN model and dynamically adjust the image contrast based on Retinex theory to obtain an enhanced user pose image.

[0033] The spatial coordinate acquisition module is used to detect the eye pixel coordinates and shoulder key point coordinates in the enhanced user pose image using a lightweight HRNet model, and convert the eye pixel coordinates into three-dimensional spatial coordinates with the center of the display as the origin by combining depth map auxiliary distance data.

[0034] The target height calculation module is used to obtain the comfortable viewing angle established by the user's initial calibration, calculate the sitting posture compensation amount based on the shoulder key point coordinates, and calculate the target height based on the sitting posture compensation amount.

[0035] An adaptive adjustment module is used to perform Kalman filtering on the three-dimensional spatial coordinates, predict the attitude change trend through a lightweight LSTM network, and generate adaptive adjustment commands based on the attitude change trend and the target height.

[0036] Furthermore, in the display height adaptive adjustment system based on user posture recognition, the target height calculation module includes the following sub-modules:

[0037] The calculation submodule is used to calculate the vertical angle between the current eye and the center of the display based on the real-time three-dimensional spatial coordinates of the eye, compare the current vertical angle with the vertical angle of the comfortable viewing angle, calculate the angle deviation, and calculate the target height of the display based on the angle deviation, the initial display height and the sitting posture compensation amount.

[0038] The constraint submodule is used to constrain the target height. If the height exceeds the range, the corresponding boundary value is taken as the final target height.

[0039] Furthermore, in the display height adaptive adjustment system based on user posture recognition, the target height calculation module includes the following sub-modules:

[0040] The submodule is used to build a lightweight LSTM network with an input layer dimension of 3, 16 hidden units in the LSTM hidden layer, a dropout probability of 0.2 in the dropout layer to prevent overfitting, and an output layer dimension of 3 in the fully connected layer, corresponding to the predicted values ​​of the three-dimensional coordinates of the eye.

[0041] The adjustment submodule is used to compare the predicted value with the filtered three-dimensional spatial coordinates, calculate the attitude change, determine the trend of user attitude change based on the sign and absolute value of the attitude change, and generate adaptive adjustment instructions based on the attitude change trend and target height.

[0042] Its beneficial effects lie in the fact that by using a camera and a ToF sensor to simultaneously collect data, combined with Retinex theory and lightweight MTCNN and HRNet models, it effectively eliminates lighting interference, improves the accuracy of posture recognition and key point detection, and ensures that the positioning errors of eye and shoulder key points are controlled within a reasonable range, providing reliable data support for height adjustment. Through initial comfort viewing angle calibration and posture compensation calculation, it adapts to the personalized needs of different users, and combines Kalman filtering and LSTM network prediction of posture change trends to achieve real-time, smooth adaptive adjustment of monitor height, avoiding over-adjustment or lag, ensuring visual comfort for users during long-term viewing, reducing cervical spine and vision-related health risks, and improving the efficiency and accuracy of monitor height adaptive adjustment. Attached Figure Description

[0043] Various other advantages and benefits will become apparent to those skilled in the art upon reading the following detailed description of preferred embodiments. The accompanying drawings are for illustrative purposes only and are not intended to limit the invention.

[0044] Figure 1 This is a schematic diagram of the first embodiment of the display height adaptive adjustment method based on user posture recognition in this invention.

[0045] Figure 2 This is a schematic diagram of a second embodiment of the display height adaptive adjustment method based on user posture recognition in this invention.

[0046] Figure 3 This is a schematic diagram of the first embodiment of the display height adaptive adjustment system based on user posture recognition in this invention. Detailed Implementation

[0047] To make the objectives, technical solutions, and advantages of this invention clearer, the invention will be further described in detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the invention.

[0048] Those skilled in the art will understand that, unless specifically stated otherwise, the singular forms "one," "an," and "this" used herein may also include the plural forms. It should be further understood that the terminology used in this specification includes the presence of features, integers, steps, operations, elements, and / or components, but does not preclude the presence or addition of one or more other features, integers, steps, operations, elements, components, and / or groups thereof.

[0049] The present invention will now be described in detail with reference to the accompanying drawings. Figure 1 As shown, a display height adaptive adjustment method based on user posture recognition includes the following steps:

[0050] Step 101: Synchronously acquire user pose images and depth maps to assist distance data using the camera and ToF sensor;

[0051] Specifically, in this embodiment, the camera is set to a 30fps frame rate, automatic exposure mode, exposure time of 10-50ms, and adaptive white balance adjustment to capture the user's upper body posture image from head to waist. The ToF sensor is set to a depth measurement range of 0.5-2.5m, covering the typical distance range between the user and the display, with a sampling frequency consistent with the camera at 30Hz, capturing depth map data between the user and the display, focusing on capturing depth information in the face and shoulder areas. The camera and ToF sensor are triggered to acquire data synchronously via the sensor driver. After each frame is acquired, a unified timestamp is added to the posture image and corresponding depth map, and stored in a temporary buffer with a capacity of 10 frames to prevent data loss. During acquisition, data validity is monitored in real time. If a frame is blurry, obscured (the obscured area exceeds 30% of the face area), or the depth map data is abnormal (the depth value exceeds the calibration range), the frame data is automatically discarded, and the next frame is quickly acquired to ensure the integrity and validity of the acquired data.

[0052] Step 102: Process the user pose image using a lightweight MTCNN model, dynamically adjust the image contrast based on Retinex theory, and obtain an enhanced user pose image.

[0053] Specifically, in this embodiment, based on the pre-trained MTCNN model, the MTCNN model is pruned and optimized, redundant convolutional layers are removed, and the three-layer structure of P-Net, R-Net, and O-Net is retained. The model input is the acquired user pose image, and the output is the filtered facial region candidate boxes. The multi-scale Retinex algorithm in Retinex theory is used to perform illumination compensation and contrast enhancement on the facial region candidate boxes. The image is converted into a grayscale image and decomposed into reflection and illumination components. The illumination component is extracted by Gaussian filtering, and the reflection component is enhanced by adaptive gain adjustment to reconstruct the enhanced user pose image. Based on the pre-trained lightweight HRNet model, a 1×1 convolutional kernel is used to replace the 3×3 convolutional kernel, and the number of channels is compressed to 1 / 2 of the original model to obtain the optimized lightweight HRNet model. The enhanced user pose image is input into the optimized lightweight HRNet model, and the output is the eye pixel coordinates and shoulder keypoint coordinates. At the same time, the confidence of the keypoints is output. Keypoints with a confidence of less than 0.9 are discarded and completed by interpolation between adjacent frames.

[0054] Load the pre-trained lightweight MTCNN (Multi-task Cascaded Convolutional Networks) model, prune and optimize the model, remove redundant convolutional layers, retain the core three-layer structure of P-Net, R-Net, and O-Net, reduce model inference latency, and ensure that the processing time per frame is ≤10ms; the model input is the original pose image acquired in step 1, and the output is the preliminary filtered candidate boxes of the facial region, with the confidence threshold set to 0.85 to filter invalid candidate boxes. For the image region corresponding to the candidate bounding box of the face area, the multi-scale Retinex algorithm (MSR) in Retinex theory is used for illumination compensation and contrast enhancement to eliminate image grayscale distortion under backlight, low light and other environments. The specific process is as follows: First, the image is converted into a grayscale image and decomposed into a reflection component, which includes pose features and an illumination component, which includes illumination interference. The illumination component is extracted by Gaussian filtering with three Gaussian kernels of different scales and standard deviations of 15, 80 and 250, respectively. Then, the reflection component is enhanced by adaptive gain adjustment, with the gain coefficient dynamically adjusted according to the mean grayscale value of the image, ranging from 1.2 to 2.0. Finally, the enhanced user pose image is reconstructed. The enhanced image must meet the following requirements: the grayscale contrast of the face area is improved by 30%-50%, the edge sharpness of key areas such as the eyes and shoulders is improved, and there is no overexposure or noise amplification. The enhanced user pose image is quality verified by calculating the peak signal-to-noise ratio (PSNR) and structural similarity index (SSIM). If PSNR ≥ 30 dB and SSIM ≥ 0.8, it is considered an effective enhanced image. If the standard is not met, the Gaussian kernel scale and gain coefficient of the Retinex algorithm are readjusted, and the contrast adjustment step is repeated until the quality requirements are met. Finally, the verified enhanced user pose image is output for subsequent keypoint detection.

[0055] Step 103: Use a lightweight HRNet model to detect the eye pixel coordinates and shoulder key point coordinates in the enhanced user pose image, and combine the depth map to assist distance data to convert the eye pixel coordinates into three-dimensional spatial coordinates with the center of the display as the origin.

[0056] Specifically, in this embodiment, the eye pixel coordinates and shoulder keypoint coordinates output by the HRNet model are filtered to remove coordinate points with insufficient confidence or abnormal positions; median filtering is used to smooth the coordinate points of consecutive frames to eliminate keypoint jitter caused by random noise; the average coordinates of the centers of the left and right pupils are calculated as the final eye pixel coordinates, and the average coordinates of the left and right shoulder vertices are calculated as the shoulder keypoint coordinates; the depth values ​​corresponding to the eye pixel coordinates are extracted from the depth map auxiliary distance data; a three-dimensional spatial coordinate system is established with the center of the display as the origin, defining the X-axis as the horizontal direction of the display, the Y-axis as the vertical direction of the display, and the Z-axis as the direction perpendicular to the display screen; based on the camera intrinsic parameters and depth values, the two-dimensional pixel coordinates of the eyes are converted into three-dimensional spatial coordinates using the pixel coordinate to three-dimensional coordinate conversion formula, and the three-dimensional spatial coordinates of the eyes are output.

[0057] Load the pre-trained lightweight HRNet and High-Resolution Net models, and optimize the model by channel compression. Replace some 3×3 convolutional kernels with 1×1 convolutional kernels, compressing the number of channels to half of the original model, ensuring that the model's inference speed meets real-time requirements, with a single-frame keypoint detection time ≤15ms. The model input is the enhanced user pose image obtained in step 2, and the output is the pixel coordinates of the eyes, the pixel coordinates of the center of the left and right pupils, denoted as (x1,y1) and (x2,y2) respectively, and the pixel coordinates of the shoulder keypoints, denoted as (x3,y3) and (x4,y4) respectively. At the same time, the confidence of each keypoint is output. Keypoints with a confidence of less than 0.9 are discarded and filled by interpolation from adjacent frames. The keypoints output by the HRNet model are filtered to remove keypoints with insufficient confidence or abnormal positions, such as eye coordinates exceeding the facial area or shoulder coordinates exceeding the upper body area. Median filtering is used to smooth the keypoint coordinates of consecutive frames with a window size of 3 frames to eliminate keypoint jitter caused by random noise, and the jitter amplitude is controlled within ±2 pixels. The average coordinates of the centers of the left and right pupils are calculated as the final eye pixel coordinates (x_eye, y_eye), and the average coordinates of the left and right shoulder vertices are calculated as the shoulder keypoint coordinates (x_shoulder, y_shoulder).

[0058] Extract the depth value d corresponding to the eye pixel coordinates (x_eye, y_eye) from the acquired depth map, which is the straight-line distance between the user's eye and the display. Establish a three-dimensional spatial coordinate system with the center of the display as the origin, defining: the X-axis is the horizontal direction of the display, positive to the right; the Y-axis is the vertical direction of the display, positive upwards; and the Z-axis is perpendicular to the display screen, positive towards the user. Based on the camera intrinsic parameters, focal length f, pixel size s, and depth value d, convert the two-dimensional pixel coordinates (x_eye, y_eye) of the eye to three-dimensional spatial coordinates (X_eye, Y_eye, Z_eye) using the pixel coordinate to three-dimensional coordinate conversion formula. The specific formulas are: X_eye = (x_eye - x0) × s × d / f, Y_eye = (y_eye - y0) × s × d / f, Z_eye = d; where (x0, y0) are the center pixel coordinates of the camera imaging plane, obtained by camera calibration, f is the camera focal length, and s is the pixel physical size; after conversion, the three-dimensional spatial coordinates of the eye (X_eye, Y_eye, Z_eye) and the coordinates of the shoulder key points (x_shoulder, y_shoulder) are output.

[0059] Step 104: Obtain the comfortable viewing angle established by the user's initial calibration, calculate the sitting posture compensation amount based on the shoulder key point coordinates, and calculate the target height based on the sitting posture compensation amount;

[0060] Specifically, this embodiment guides the user to perform a comfortable viewing angle calibration. It collects the real-time 3D spatial coordinates of the eyes and the real-time coordinates of key shoulder points, records the initial monitor height, and calculates the vertical angle between the eyes and the monitor center. The range of the vertical angle for the comfortable viewing angle is set to ±3°. Based on a comparison of the real-time shoulder key point coordinates, the vertical posture offset is calculated, and a posture compensation amount is calculated based on this offset. Using the real-time 3D spatial coordinates of the eyes, the current vertical angle between the eyes and the monitor center is calculated. This angle is compared to the vertical angle for the comfortable viewing angle to calculate the angle deviation. Based on the angle deviation, the initial monitor height, and the posture compensation amount, the target monitor height is calculated. The target height is constrained; if it exceeds the range, the corresponding boundary value is taken as the final target height.

[0061] Upon initial system startup, the system guides the user through a comfortable viewing angle calibration. After adjusting the monitor to their most comfortable viewing height and angle, the user maintains a natural sitting posture with their back straight and eyes looking straight ahead at the center of the monitor. The system collects the three-dimensional spatial coordinates of the eyes at this moment, denoted as (X0, Y0, Z0), the coordinates of the key points of the shoulders, denoted as (x0_shoulder, y0_shoulder), and records the monitor height at this moment, denoted as H0. The viewing angle in this state is defined as the comfortable viewing angle, and the comfortable viewing angle parameter is calculated as the vertical angle θ0 between the eyes and the center of the monitor, θ0 = arctan(|Y0| / Z0). The range of the vertical angle of the comfortable viewing angle is set to θ0 ± 3°, which serves as the reference for subsequent height adjustments.

[0062] Based on the real-time shoulder keypoint coordinates (x_shoulder, y_shoulder) output in step 3, compare them with the initially calibrated shoulder keypoint coordinates (x0_shoulder, y0_shoulder) to calculate the sitting posture offset. Specifically, calculate the vertical sitting posture offset Δy = y_shoulder - y0_shoulder. If Δy > 0, it indicates that the user is leaning forward; if Δy < 0, it indicates that the user is leaning backward. Calculate the sitting posture compensation ΔH_compensate based on the sitting posture offset Δy. The compensation formula is: ΔH_compensate = k × Δy, where k is the compensation coefficient, calibrated experimentally, with a value range of 0.3-0.5, unit: mm / pixel. This is used to correct the eye height offset under different sitting postures to ensure that the target height is adapted to the current sitting posture.

[0063] First, based on the real-time three-dimensional spatial coordinates of the eyes (X_eye, Y_eye, Z_eye), calculate the vertical angle θ between the current eye and the center of the monitor = arctan(|Y_eye| / Z_eye); compare the current vertical angle θ with the vertical angle θ0 of the comfortable viewing angle, and calculate the angle deviation Δθ = θ - θ0; based on the angle deviation Δθ, the initial monitor height H0, and the posture compensation amount ΔH_compensate, calculate the monitor target height H_target, specifically: H_target = H0 + (Z_eye ×tan(θ0) - Y_eye) + ΔH_compensate; where (Z_eye × tan(θ0) - Y_eye) is the height adjustment amount based on the comfortable viewing angle, used to adjust the eye height to within the comfortable viewing angle range; after calculation, constrain the target height H_target to ensure that it is within the monitor adjustment range, i.e., H_min ≤ H_target ≤ H_max, where H_min is the minimum adjustment height of the monitor, and H_max is the maximum adjustment height of the monitor. If it exceeds the range, the corresponding boundary value is taken as the final target height.

[0064] Step 105: Perform Kalman filtering on the three-dimensional spatial coordinates, predict the attitude change trend through a lightweight LSTM network, and generate adaptive adjustment commands based on the attitude change trend and target height.

[0065] Specifically, in this embodiment, a lightweight LSTM network is constructed with an input layer dimension of 3, 16 hidden units in the LSTM hidden layer, and a dropout probability of 0.2 in the dropout layer to prevent overfitting. The output layer dimension of the fully connected layer is 3, corresponding to the predicted value of the three-dimensional coordinates of the eye. The predicted value is compared with the filtered three-dimensional spatial coordinates to calculate the pose change. Based on the sign and absolute value of the pose change, the trend of the user's pose change is determined. An adaptive adjustment command is generated based on the pose change trend and the target height.

[0066] The three-dimensional spatial coordinates (X_eye, Y_eye, Z_eye) output from step 3 are processed using Kalman filtering to eliminate random noise, such as the influence of slight user movement or sensor measurement errors on the coordinate data. The Kalman filter state equation and observation equation are established: the state equation is X_k = A × X_{k-1} + B × u_k + w_k, and the observation equation is Z_k = H × X_k + v_k; where X_k is the state vector at time k, containing the three-dimensional coordinates and velocity of the eye, A is the state transition matrix, B is the control matrix, u_k is the control variable, initially set to 0, w_k is the process noise, and the variance is set to Q=diag([1e-4,1e-4,1e-4,1e-6,1e-6,1e-6]), Z_k is the observation vector at time k, the three-dimensional coordinates of the eye output in step 3, H is the observation matrix, v_k is the observation noise, and the variance is set to R=diag([1e-3,1e-3,1e-3]); through the prediction step and update step of Kalman filtering, the filtered three-dimensional spatial coordinates of the eye (X_eye_filter, Y_eye_filter, Z_eye_filter) are output to ensure the stability of the coordinate data, with a fluctuation range ≤±1mm.

[0067] A lightweight LSTM (Long Short-Term Memory) network is constructed to predict changes in user pose. The network input consists of filtered eye 3D coordinate data from the last 10 frames, with a time series length of 10. The network structure is as follows: Input layer, dimension 3 → 1 LSTM hidden layer with 16 hidden units → Dropout layer with a dropout probability of 0.2 to prevent overfitting → Fully connected layer, output layer, dimension 3, corresponding to the predicted eye 3D coordinates (X_eye_pred, Y_eye_pred, Z_eye_pred) for the next frame. The predicted eye 3D coordinates are compared with the current filtered coordinates to calculate the pose change ΔX_pred = X_eye_pred - X_eye_filter, ΔY_pred = Y_eye_pred - Y_eye_filter, ΔZ_pred = Z_eye_pred - Z_eye_filter determines the trend of user posture changes based on the sign and absolute value of ΔY_pred. For example, if ΔY_pred>0 and the absolute value>0.5mm, it means that the user's eye height will rise, and the monitor needs to be adjusted upward in advance.

[0068] Combining the target height H_target calculated in step 4, the attitude change trend predicted by the LSTM network, and ΔY_pred, an adaptive adjustment command for the display height is generated; specifically: first, the adjustment deviation ΔH = H_target - H_current represents the current height of the display. Based on the attitude change trend ΔY_pred, the adjustment deviation is corrected: if ΔY_pred and ΔH have the same sign (both positive), it indicates the target height is rising and the user's eye will continue to rise, so the adjustment speed is increased, set to v1 = 2mm / frame; if ΔY_pred and ΔH have opposite signs, the adjustment speed is decreased, set to v2 = 1mm / frame to avoid over-adjustment. The format of the adjustment command is: [adjustment direction, up / down, adjustment speed, adjustment amplitude], where the adjustment amplitude is the absolute value of ΔH, and the adjustment direction is determined by the sign of ΔH (ΔH > 0 for up, ΔH < 0 for down). The adjustment command is sent to the display's height adjustment actuator, such as an electric lifting bracket. The actuator adaptively adjusts according to the command, collecting the current height of the display in real time during the adjustment process. When the deviation between H_current and H_target is ≤ ±0.5mm, the adjustment stops. Simultaneously, the adjustment command and current attitude data are stored in the system log for subsequent algorithm optimization.

[0069] Its beneficial effects lie in the fact that by using a camera and a ToF sensor to simultaneously collect data, combined with Retinex theory and lightweight MTCNN and HRNet models, it effectively eliminates lighting interference, improves the accuracy of posture recognition and key point detection, and ensures that the positioning errors of eye and shoulder key points are controlled within a reasonable range, providing reliable data support for height adjustment. Through initial comfort viewing angle calibration and posture compensation calculation, it adapts to the personalized needs of different users, and combines Kalman filtering and LSTM network prediction of posture change trends to achieve real-time, smooth adaptive adjustment of monitor height, avoiding over-adjustment or lag, ensuring visual comfort for users during long-term viewing, reducing cervical spine and vision-related health risks, and improving the efficiency and accuracy of monitor height adaptive adjustment.

[0070] Please see Figure 2 In the display height adaptive adjustment method based on user pose recognition, the lightweight HRNet model is used to detect the eye pixel coordinates and shoulder key point coordinates in the enhanced user pose image. The eye pixel coordinates are then converted into three-dimensional spatial coordinates with the display center as the origin by combining depth map-assisted distance data. This includes the following steps:

[0071] Step 201: Filter the eye pixel coordinates and shoulder keypoint coordinates output by the HRNet model, and remove coordinate points with insufficient confidence or abnormal position; use median filtering to smooth the coordinate points of consecutive frames to eliminate keypoint jitter caused by random noise.

[0072] Step 202: Calculate the average coordinates of the centers of the left and right pupils as the final eye pixel coordinates, and calculate the average coordinates of the left and right shoulder vertices as the shoulder keypoint coordinates.

[0073] Step 203: Extract the depth values ​​corresponding to the eye pixel coordinates from the auxiliary distance data of the depth map; establish a three-dimensional spatial coordinate system with the center of the display as the origin, and define the X-axis as the horizontal direction of the display, the Y-axis as the vertical direction of the display, and the Z-axis as the direction perpendicular to the display screen.

[0074] Step 204: Based on the camera intrinsic parameters and depth values, convert the two-dimensional pixel coordinates of the eye into three-dimensional spatial coordinates using the pixel coordinate to three-dimensional coordinate conversion formula, and output the three-dimensional spatial coordinates of the eye.

[0075] The above describes embodiments of the display height adaptive adjustment method based on user posture recognition of the present invention. Please refer to [link / reference]. Figure 3 In a user posture recognition-based adaptive display height adjustment system, the system includes the following modules:

[0076] The posture image acquisition module is used to simultaneously acquire user posture images and depth maps to assist distance data through a camera and a ToF sensor;

[0077] The image data enhancement module is used to process the user pose image using a lightweight MTCNN model and dynamically adjust the image contrast based on Retinex theory to obtain an enhanced user pose image.

[0078] The spatial coordinate acquisition module is used to detect the eye pixel coordinates and shoulder key point coordinates in the enhanced user pose image using a lightweight HRNet model, and convert the eye pixel coordinates into three-dimensional spatial coordinates with the center of the display as the origin by combining depth map auxiliary distance data.

[0079] The target height calculation module is used to obtain the comfortable viewing angle established by the user's initial calibration, calculate the sitting posture compensation amount based on the shoulder key point coordinates, and calculate the target height based on the sitting posture compensation amount.

[0080] An adaptive adjustment module is used to perform Kalman filtering on the three-dimensional spatial coordinates, predict the attitude change trend through a lightweight LSTM network, and generate adaptive adjustment commands based on the attitude change trend and the target height.

[0081] Its beneficial effects lie in the fact that by using a camera and a ToF sensor to simultaneously collect data, combined with Retinex theory and lightweight MTCNN and HRNet models, it effectively eliminates lighting interference, improves the accuracy of posture recognition and key point detection, and ensures that the positioning errors of eye and shoulder key points are controlled within a reasonable range, providing reliable data support for height adjustment. Through initial comfort viewing angle calibration and posture compensation calculation, it adapts to the personalized needs of different users, and combines Kalman filtering and LSTM network prediction of posture change trends to achieve real-time, smooth adaptive adjustment of monitor height, avoiding over-adjustment or lag, ensuring visual comfort for users during long-term viewing, reducing cervical spine and vision-related health risks, and improving the efficiency and accuracy of monitor height adaptive adjustment.

[0082] The foregoing has shown and described the basic principles, main features, and advantages of the present invention. Those skilled in the art should understand that the present invention is not limited to the above embodiments. The embodiments and descriptions in the specification are merely preferred examples and are not intended to limit the invention. Various changes and modifications can be made to the invention without departing from its spirit and scope, and all such changes and modifications fall within the scope of the present invention as claimed. The scope of protection of the present invention is defined by the appended technical solutions and equivalents.

Claims

1. A display height adaptive adjustment method based on user posture recognition, characterized in that, The display height adaptive adjustment method includes the following steps: The user's posture image and depth map are simultaneously acquired by the camera and ToF sensor to assist in distance data acquisition. The user pose image is processed by a lightweight MTCNN model, and the image contrast is dynamically adjusted based on Retinex theory to obtain an enhanced user pose image. The lightweight HRNet model is used to detect the eye pixel coordinates and shoulder key point coordinates in the enhanced user pose image. The eye pixel coordinates are converted into three-dimensional spatial coordinates with the center of the display as the origin by combining depth map auxiliary distance data. Obtain the comfortable viewing angle established by the user's initial calibration, calculate the sitting posture compensation amount based on the shoulder key point coordinates, and calculate the target height based on the sitting posture compensation amount; Kalman filtering is applied to the three-dimensional spatial coordinates, and the attitude change trend is predicted through a lightweight LSTM network. Adaptive adjustment commands are then generated based on the attitude change trend and the target height.

2. The display height adaptive adjustment method based on user posture recognition as described in claim 1, characterized in that, The process of processing the user pose image using a lightweight MTCNN model, and dynamically adjusting the image contrast based on Retinex theory to obtain an enhanced user pose image, includes: Based on the pre-trained MTCNN model, the MTCNN model is pruned and optimized by removing redundant convolutional layers and retaining the three-layer structure of P-Net, R-Net and O-Net. The model input is the collected user pose image and the output is the filtered candidate boxes of facial regions. The multi-scale Retinex algorithm in Retinex theory is used to perform illumination compensation and contrast enhancement on candidate bounding boxes of facial regions. The image is converted into a grayscale image and decomposed into reflection and illumination components. The illumination component is extracted by Gaussian filtering, and the reflection component is enhanced by adaptive gain adjustment. The enhanced user pose image is then reconstructed.

3. The display height adaptive adjustment method based on user posture recognition as described in claim 1, characterized in that, The process of using a lightweight HRNet model to detect eye pixel coordinates and shoulder keypoint coordinates in the enhanced user pose image, and combining depth map-assisted distance data to convert the eye pixel coordinates into three-dimensional spatial coordinates with the display center as the origin, includes: Based on the pre-trained lightweight HRNet model, a 1×1 convolutional kernel is used to replace the 3×3 convolutional kernel, and the number of channels is compressed to half of the original model, resulting in an optimized lightweight HRNet model. The enhanced user pose image is input into the optimized lightweight HRNet model, which outputs eye pixel coordinates and shoulder keypoint coordinates, as well as the confidence score of the keypoints. Keypoints with a confidence score below 0.9 are discarded and filled in by interpolation from adjacent frames.

4. The display height adaptive adjustment method based on user posture recognition as described in claim 1, characterized in that, The process of using a lightweight HRNet model to detect eye pixel coordinates and shoulder keypoint coordinates in the enhanced user pose image, and combining depth map-assisted distance data to convert the eye pixel coordinates into three-dimensional spatial coordinates with the display center as the origin, includes: The eye pixel coordinates and shoulder keypoint coordinates output by the HRNet model are filtered to remove coordinate points with insufficient confidence or abnormal position; median filtering is used to smooth the coordinate points of consecutive frames to eliminate keypoint jitter caused by random noise. The average coordinates of the centers of the left and right pupils are calculated as the final eye pixel coordinates, and the average coordinates of the vertices of the left and right shoulders are calculated as the shoulder keypoint coordinates. Extract the depth values ​​corresponding to the eye pixel coordinates from the auxiliary distance data of the depth map; establish a three-dimensional spatial coordinate system with the center of the display as the origin, defining the X-axis as the horizontal direction of the display, the Y-axis as the vertical direction of the display, and the Z-axis as the direction perpendicular to the display screen; Based on the camera's intrinsic parameters and depth values, the two-dimensional pixel coordinates of the eye are converted into three-dimensional spatial coordinates using a formula that converts pixel coordinates to three-dimensional coordinates, and the three-dimensional spatial coordinates of the eye are output.

5. The display height adaptive adjustment method based on user posture recognition as described in claim 1, characterized in that, The process of obtaining the user's initial calibration-established comfortable viewing angle, calculating the sitting posture compensation amount based on the shoulder key point coordinates, and calculating the target height based on the sitting posture compensation amount includes: Guide users to perform comfortable viewing angle calibration, collect real-time three-dimensional spatial coordinates of the eyes and real-time key point coordinates of the shoulders, record the initial display height, calculate the vertical angle between the eyes and the center of the display, and set the vertical angle range of the comfortable viewing angle to ±3°. The vertical sitting posture offset is calculated by comparing the real-time shoulder key point coordinates with the actual shoulder key point coordinates, and the sitting posture compensation is calculated based on the sitting posture offset.

6. The display height adaptive adjustment method based on user posture recognition as described in claim 1, characterized in that, The process of obtaining the user's initial calibration-established comfortable viewing angle, calculating the sitting posture compensation amount based on the shoulder key point coordinates, and calculating the target height based on the sitting posture compensation amount includes: Based on the real-time three-dimensional spatial coordinates of the eyes, calculate the vertical angle between the current eyes and the center of the monitor. Compare the current vertical angle with the vertical angle of the comfortable viewing angle to calculate the angle deviation. Based on the angle deviation, the initial monitor height, and the posture compensation amount, calculate the target height of the monitor. The target height is constrained; if it exceeds the range, the corresponding boundary value is taken as the final target height.

7. The display height adaptive adjustment method based on user posture recognition as described in claim 1, characterized in that, The process of performing Kalman filtering on the three-dimensional spatial coordinates, predicting the attitude change trend through a lightweight LSTM network, and generating adaptive adjustment commands based on the attitude change trend and target height includes: A lightweight LSTM network is constructed with an input layer dimension of 3, 16 hidden units in the LSTM hidden layer, and a dropout probability of 0.2 in the dropout layer to prevent overfitting. The output layer dimension of the fully connected layer is 3, corresponding to the predicted value of the three-dimensional coordinates of the eye. The predicted value is compared with the filtered three-dimensional spatial coordinates to calculate the attitude change. Based on the sign and absolute value of the attitude change, the trend of the user's attitude change is determined. An adaptive adjustment command is generated based on the attitude change trend and the target height.

8. A display height adaptive adjustment system based on user posture recognition, characterized in that, The display height adaptive adjustment system includes the following modules: The posture image acquisition module is used to simultaneously acquire user posture images and depth maps to assist distance data through a camera and a ToF sensor; The image data enhancement module is used to process the user pose image using a lightweight MTCNN model and dynamically adjust the image contrast based on Retinex theory to obtain an enhanced user pose image. The spatial coordinate acquisition module is used to detect the eye pixel coordinates and shoulder key point coordinates in the enhanced user pose image using a lightweight HRNet model, and convert the eye pixel coordinates into three-dimensional spatial coordinates with the center of the display as the origin by combining depth map auxiliary distance data. The target height calculation module is used to obtain the comfortable viewing angle established by the user's initial calibration, calculate the sitting posture compensation amount based on the shoulder key point coordinates, and calculate the target height based on the sitting posture compensation amount. An adaptive adjustment module is used to perform Kalman filtering on the three-dimensional spatial coordinates, predict the attitude change trend through a lightweight LSTM network, and generate adaptive adjustment commands based on the attitude change trend and the target height.

9. The display height adaptive adjustment system based on user posture recognition as described in claim 8, characterized in that, The target height calculation module includes the following sub-modules: The calculation submodule is used to calculate the vertical angle between the current eye and the center of the display based on the real-time three-dimensional spatial coordinates of the eye, compare the current vertical angle with the vertical angle of the comfortable viewing angle, calculate the angle deviation, and calculate the target height of the display based on the angle deviation, the initial display height and the sitting posture compensation amount. The constraint submodule is used to constrain the target height. If the height exceeds the range, the corresponding boundary value is taken as the final target height.

10. The display height adaptive adjustment system based on user posture recognition as described in claim 8, characterized in that, The target height calculation module includes the following sub-modules: The submodule is used to build a lightweight LSTM network with an input layer dimension of 3, 16 hidden units in the LSTM hidden layer, a dropout probability of 0.2 in the dropout layer to prevent overfitting, and an output layer dimension of 3 in the fully connected layer, corresponding to the predicted values ​​of the three-dimensional coordinates of the eye. The adjustment submodule is used to compare the predicted value with the filtered three-dimensional spatial coordinates, calculate the attitude change, determine the trend of user attitude change based on the sign and absolute value of the attitude change, and generate adaptive adjustment instructions based on the attitude change trend and target height.

Images