Display screen direction dynamic adjusting system and adjusting method based on user behaviors

Abstract

The invention relates to a display screen direction dynamic adjusting system and method based on user behaviors, and the system comprises a movable robot platform which is configured to achieve indoor autonomous movement through a bottom drive mechanism which comprises a high-precision motion assembly suitable for various ground materials; the six-degree-of-freedom mechanical arm module is vertically mounted at the top of the robot platform, and a display screen mounting interface is arranged at the tail end; the intelligent sensing module, the integrated infrared sensor array, the nine-axis inertial measurement unit and the depth vision sensor are configured to capture user biological characteristics and environment space data in real time, and through the movable robot platform and the six-degree-of-freedom mechanical arm module, autonomous movement and three-dimensional pose accurate adjustment of a display screen are achieved; and in combination with a machine learning algorithm and a sight tracking technology of the adaptive control module, highly autonomous and accurate user interaction experience is provided.

Description

Technical Field

[0001] The present invention relates to the field of display screen adjustment, and in particular to a dynamic display screen orientation adjustment system and method based on user behavior. Background Art

[0002] As an advanced human-computer interaction means, eye tracking technology has been gradually applied to devices such as computers, allowing users to control interface elements such as software icons through eye movements. This technology enables users to simply gaze at the icons or options on the display screen, and the eye tracking function can quickly capture this action and accurately position the focus on the gazed position, thus eliminating the need for manual clicking or swiping operations. However, this traditional eye tracking technology is mainly limited to the control of computer software icons, and its application scenarios are relatively single.

[0003] In actual different working scenarios, users often need to face the display screen for a long time to perform operations, and their positions and postures often change involuntarily. Such changes in position and posture are likely to cause changes in the relative position between the eyes and the display screen, thereby affecting the accuracy of eye tracking. Due to the lack of the ability to dynamically adjust the display screen, the traditional eye tracking technology is difficult to adapt to such posture changes, resulting in the users may need to frequently adjust their postures or the position of the display screen during use to maintain the accuracy of eye tracking, which not only affects the work efficiency of the users, but also easily brings unnecessary physical burdens to the users.

[0004] Therefore, we propose a dynamic display screen orientation adjustment system and method based on user behavior, which not only retains the original advantages of eye tracking technology, but also realizes the dynamic adjustment function of the display screen, and can dynamically adjust the pitch angle, azimuth angle and height of the display screen according to the user's posture and eye position to ensure that the screen content always remains within the user's best line of sight. Summary of the Invention

[0005] The purpose of the present invention is to overcome the deficiencies of the prior art, adapt to the actual needs, and provide a dynamic display screen orientation adjustment system and method based on user behavior to solve the technical problems that in actual different working scenarios, users often need to face the display screen for a long time to perform operations, and their positions and postures often change involuntarily. Such changes in position and posture are likely to cause changes in the relative position between the eyes and the display screen, thereby affecting the accuracy of eye tracking. Due to the lack of the ability to dynamically adjust the display screen, the traditional eye tracking technology is difficult to adapt to such posture changes, resulting in the users may need to frequently adjust their postures or the position of the display screen during use to maintain the accuracy of eye tracking, which not only affects the work efficiency of the users, but also may bring unnecessary physical burdens to the users.

[0006] To achieve the objectives of the present invention, the technical solution adopted by the present invention is as follows: Design a dynamic display screen orientation adjustment system based on user behavior, including:

[0007] A mobile robot platform configured to achieve autonomous movement indoors through a bottom drive mechanism, and the drive mechanism includes a high-precision motion component adapted to various floor materials;

[0008] A six-degree-of-freedom robotic arm module vertically installed on the top of the robot platform, with a display screen mounting interface at the end;

[0009] An intelligent sensing module integrating an infrared sensor array, a nine-axis inertial measurement unit, and a depth vision sensor, configured to capture user biometric and environmental space data in real time;

[0010] An adaptive control module including a multi-core processor and a motion control unit, where the multi-core processor is configured to run machine learning algorithms to analyze user body language features, and the motion control unit synchronously generates three-dimensional space trajectory commands;

[0011] Among them, the robot platform is navigated to the user-specified area through the drive mechanism;

[0012] The six-degree-of-freedom robotic arm is used to perform three-dimensional pose adjustment of the display screen, with a rotation accuracy ≤ 0.1° and a translational positioning error < 1 mm;

[0013] Based on the real-time data stream of the intelligent sensing module, the pitch angle and azimuth angle of the display screen are dynamically corrected to keep the deviation between the screen normal vector and the user's visual axis direction < 3°.

[0014] Preferably, the high-precision motion component adopts an omnidirectional wheel system and lidar SLAM combined navigation, and its path planning module is configured as:

[0015] Establish a cost map including dynamic obstacle prediction and generate an optimal movement path through the A* algorithm;

[0016] When it is detected that the user's line-of-sight focus deviation exceeds a preset threshold, an autonomous obstacle avoidance replanning mechanism is triggered.

[0017] Preferably, the six-degree-of-freedom robotic arm module includes:

[0018] A series joint structure, with each joint configured with a harmonic reducer and an absolute encoder;

[0019] The end effector is provided with a three-axis pan-tilt mechanism, and its rotation range covers an azimuth angle of ±180° and a pitch angle of ±90°;

[0020] A force / torque sensor is integrated at the wrist of the robotic arm to monitor external disturbances in real time and trigger active compliance control.

[0021] Preferably, the intelligent perception module further includes:

[0022] A millimeter-wave radar array, configured circumferentially on the robot platform for detecting moving objects within 5 meters;

[0023] A binocular stereo vision module, equipped with an infrared structured light projector, achieving a three-dimensional reconstruction accuracy of ±2 mm within a distance of 0.1 - 5 meters;

[0024] A surface electromyography sensor, configured to capture the user's upper limb movement intention signal.

[0025] Preferably, the adaptive control module includes:

[0026] A behavior recognition sub-module, which processes 20 consecutive frames of skeletal key point data using a spatio-temporal convolutional neural network;

[0027] A gaze tracking sub-module, which calculates the three-dimensional gaze point coordinates by combining the corneal reflection vector and the pupil center coordinates;

[0028] A dynamic weight fusion unit, which performs Bayesian probability fusion on the confidence levels of the head pose, gesture commands, and gaze focus.

[0029] Preferably, it further includes a multi-modal interaction interface, including:

[0030] An augmented reality projection device, which superimposes a virtual control panel within the user's field of view;

[0031] A voice interaction unit, which supports the intention recognition of natural language commands;

[0032] A tactile feedback device, integrated at the operation terminal of the robotic arm, providing operating force feedback guidance.

[0033] Preferably, it further includes an implementation scenario adaptation strategy:

[0034] When it detects that the user performs a standing-sitting position conversion, it triggers a height compensation algorithm to keep the height difference between the center point of the display screen and the user's eyes within the range of ±50 mm;

[0035] In the multi-person meeting mode, it starts a periodic scanning protocol and dynamically adjusts the orientation of the display screen according to the sound source localization of the speaker.

[0036] Preferably, it further includes a three-dimensional space modeling engine:

[0037] It uses a ToF camera array to construct an indoor point cloud model with a resolution of 512×512×256 voxels;

[0038] An environmental semantic segmentation unit, which identifies the semantic labels of furniture and electrical appliances categories and establishes a dynamic passable area map;

[0039] User-defined interface, supporting the annotation of multiple preferred parking poses in the 3D model.

[0040] Preferably, it also includes a safety protection mechanism:

[0041] When the acceleration of the end of the robotic arm exceeds 4m / s 2 Activate dynamic damping control;

[0042] During the movement of the robot, if a sudden obstacle appears on the detected path, the emergency braking time ≤ 200ms.

[0043] The method for dynamically adjusting the display screen direction based on user behavior, including the system for dynamically adjusting the display screen direction based on user behavior, includes the following steps:

[0044] S1. Start the mobile robot platform, initialize the bottom drive mechanism and the high-precision motion components, initialize the six-degree-of-freedom robotic arm module, check the status of each joint harmonic reducer and the absolute encoder, start the intelligent perception module, including the infrared sensor array, the nine-axis inertial measurement unit, the depth vision sensor, etc., ensure the normal real-time data stream, start the adaptive control module, the multi-core processor loads the machine learning algorithm, and the motion control unit is ready to receive instructions;

[0045] S2. The user specifies the moving area of the robot through the multi-modal interaction interface, such as the augmented reality projection device and the voice interaction unit. The adaptive control module receives the user's instructions, generates the optimal moving path through the path planning module, and the robot platform moves to the specified area according to the navigation instructions by using the omnidirectional wheel system and the lidar SLAM combined navigation;

[0046] S3. The user indicates the desired pose of the display screen through the multi-modal interaction interface or body language, such as gestures and head postures. The behavior recognition sub-module of the adaptive control module processes the user's body language features, the gaze tracking sub-module calculates the coordinates of the user's fixation point, and the motion control unit generates a three-dimensional space trajectory instruction according to the user's instructions, and the six-degree-of-freedom robotic arm executes the three-dimensional pose adjustment of the display screen to ensure that the rotation accuracy ≤ 0.1° and the translation positioning error < 1mm;

[0047] S4. The intelligent perception module captures the user's biometric features and environmental space data in real time, including the millimeter-wave radar array detecting moving objects, the binocular stereo vision module realizing three-dimensional reconstruction, and the surface electromyography sensor capturing the user's upper limb movement intention. The adaptive control module dynamically corrects the pitch angle and azimuth angle of the display screen according to the real-time data stream to keep the deviation between the normal vector of the screen and the user's visual axis direction < 3°;

[0048] S5. When it is detected that the user performs a standing-sitting conversion, trigger the height compensation algorithm to keep the height difference between the center point of the display screen and the user's eyes within the range of ±50 mm. In the multi-person meeting mode, start the periodic scanning protocol and dynamically adjust the orientation of the display screen according to the sound source localization of the speaker.

[0049] S6. Use the ToF camera array to construct an indoor point cloud model with a resolution of 512×512×256 voxels. The environmental semantic segmentation unit identifies the semantic labels of furniture and electrical appliances categories and establishes a dynamic passable area map. The user can mark multiple preferred stopping poses in the 3D model through a custom interface.

[0050] S7. When the acceleration at the end of the robotic arm exceeds 4 m / s 2 ², activate the dynamic damping control. During the movement of the robot, if a sudden obstacle appears on the detected path, perform an emergency brake with a time ≤ 200 ms.

[0051] Compared with the prior art, the beneficial effects of the present invention are as follows:

[0052] 1. The present invention realizes the autonomous movement and precise three-dimensional pose adjustment of the display screen through a mobile robot platform and a six-degree-of-freedom robotic arm module. Combining the machine learning algorithm and eye tracking technology of the adaptive control module, it provides a highly autonomous and accurate user interaction experience.

[0053] 2. The intelligent perception module of the present invention integrates multiple sensors to capture the user's biometric characteristics and environmental space data in real time, enabling the system to comprehensively perceive the changes in the user's posture, intention, and the surrounding environment, and dynamically correct the pitch angle and azimuth angle of the display screen to maintain the best line of sight direction.

[0054] 3. The present invention supports multiple interaction methods such as augmented reality projection, voice interaction, and tactile feedback to meet the different needs of users. At the same time, it provides a user-defined interface that allows marking preferred stopping poses in the 3D model to achieve personalized customization.

[0055] 4. The present invention has the ability of intelligent scene recognition, can recognize scenes such as user posture conversion and multi-person meetings, and automatically adjust the position of the display screen. At the same time, it is equipped with a safety protection mechanism to ensure the safety and stability during the operation process. BRIEF DESCRIPTION OF THE DRAWINGS

[0056] Figure 1 is a system schematic diagram of the present invention;

[0057] Figure 2 is a schematic diagram of the high-precision motion component of the present invention;

[0058] Figure 3 is a schematic diagram of the six-degree-of-freedom robotic arm module of the present invention;

[0059] Figure 4 Schematic diagram of the intelligent perception module of the present invention;

[0060] Figure 5 Schematic diagram of the adaptive control module of the present invention;

[0061] Figure 6 Schematic diagram of the process of the present invention. Detailed implementation manners

[0062] The present invention will be further described below in conjunction with the accompanying drawings and embodiments:

[0063] A dynamic display screen direction adjustment system based on user behavior, see Figures 1 to 6 , including:

[0064] A mobile robot platform configured to achieve autonomous movement indoors through a bottom drive mechanism, and the drive mechanism includes high-precision motion components adapted to various floor materials;

[0065] A six-degree-of-freedom robotic arm module vertically installed on the top of the robot platform, with a display screen mounting interface at the end;

[0066] An intelligent perception module integrating an infrared sensor array, a nine-axis inertial measurement unit, and a depth vision sensor, configured to capture user biometric and environmental space data in real time;

[0067] An adaptive control module including a multi-core processor and a motion control unit. The multi-core processor is configured to run machine learning algorithms to analyze user body language features, and the motion control unit synchronously generates three-dimensional space trajectory commands;

[0068] Among them, the robot platform is navigated to the user-specified area through the drive mechanism;

[0069] The six-degree-of-freedom robotic arm is used to perform three-dimensional pose adjustment of the display screen, with a rotation accuracy ≤ 0.1° and a translational positioning error < 1 mm;

[0070] Based on the real-time data stream of the intelligent perception module, the pitch angle and azimuth angle of the display screen are dynamically corrected to keep the deviation between the screen normal vector and the user's visual axis direction < 3°.

[0071] Specifically, the high-precision motion components adopt an omnidirectional wheel system and lidar SLAM combined navigation, and its path planning module is configured as:

[0072] Establish a cost map including dynamic obstacle prediction, and generate an optimal movement path through the A* algorithm;

[0073] When it is detected that the deviation of the user's line-of-sight focus exceeds a preset threshold, an autonomous obstacle avoidance replanning mechanism is triggered.

[0074] Specifically, the six-degree-of-freedom robotic arm module includes:

[0075] A serial joint structure, with each joint equipped with a harmonic reducer and an absolute encoder;

[0076] The end effector is provided with a three-axis pan-tilt mechanism, and its rotation range covers azimuth angles of ±180° and pitch angles of ±90°;

[0077] The force / torque sensor is integrated in the wrist of the robotic arm to monitor external disturbances in real time and trigger active compliance control.

[0078] Furthermore, the intelligent perception module further includes:

[0079] A millimeter-wave radar array, configured circumferentially on the robot platform, for detecting moving objects within a range of 5 meters;

[0080] A binocular stereo vision module, equipped with an infrared structured light projector, achieving a three-dimensional reconstruction accuracy of ±2 mm within a distance range of 0.1 - 5 meters;

[0081] Surface electromyography sensors, configured to capture the motion intention signals of the user's upper limb.

[0082] Even further, the adaptive control module includes:

[0083] A behavior recognition sub-module, which processes 20 consecutive frames of skeletal key point data using a spatio-temporal convolutional neural network;

[0084] A gaze tracking sub-module, which calculates the three-dimensional gaze point coordinates by combining the corneal reflection vector and the pupil center coordinates;

[0085] A dynamic weight fusion unit, which performs Bayesian probability fusion on the confidence levels of the head pose, gesture commands, and gaze focus.

[0086] It should be noted that it also includes a multi-modal interaction interface, including:

[0087] An augmented reality projection device, which superimposes a virtual control panel within the user's field of view;

[0088] A voice interaction unit, which supports the intent recognition of natural language commands;

[0089] A tactile feedback device, integrated at the operation terminal of the robotic arm, providing operating force feedback guidance.

[0090] It should be noted that it also includes an implementation scenario adaptation strategy:

[0091] When it detects that the user performs a standing-sitting position conversion, it triggers a height compensation algorithm to keep the height difference between the center point of the display screen and the user's eyes within a range of ±50 mm;

[0092] In the multi-person meeting mode, start the periodic scanning protocol and dynamically adjust the display screen orientation according to the speaker sound source localization.

[0093] It is worth introducing that it also includes a three-dimensional space modeling engine:

[0094] Use a ToF camera array to construct an indoor point cloud model with a resolution of 512×512×256 voxels;

[0095] The environmental semantic segmentation unit identifies the semantic labels of furniture and electrical appliances categories and establishes a dynamic passable area map;

[0096] The user-defined interface supports marking multiple preferred parking poses in the three-dimensional model.

[0097] It is worth emphasizing that it also includes a safety protection mechanism:

[0098] When the acceleration at the end of the robotic arm exceeds 4m / s 2 Activate the dynamic damping control;

[0099] During the movement of the robot, if a sudden obstacle appears on the detected path, the emergency braking time ≤ 200ms.

[0100] The method for dynamically adjusting the display screen direction based on user behavior, including the system for dynamically adjusting the display screen direction based on user behavior, includes the following steps:

[0101] S1. Start the mobile robot platform, initialize the bottom drive mechanism and the high-precision motion components, initialize the six-degree-of-freedom robotic arm module, check the status of each joint harmonic reducer and the absolute encoder, start the intelligent perception module, including the infrared sensor array, the nine-axis inertial measurement unit, the depth vision sensor, etc., ensure the normal real-time data stream, start the adaptive control module, the multi-core processor loads the machine learning algorithm, and the motion control unit is ready to receive instructions;

[0102] S2. The user specifies the robot movement area through the multi-modal interaction interface, such as the augmented reality projection device and the voice interaction unit. The adaptive control module receives the user instructions, generates the optimal movement path through the path planning module, and the robot platform moves to the specified area according to the navigation instructions using the omnidirectional wheel system and the lidar SLAM combined navigation;

[0103] S3. The user indicates the desired pose of the display screen through the multi-modal interaction interface or body language, such as gestures and head postures. The behavior recognition sub-module of the adaptive control module processes the user body language features, the gaze tracking sub-module calculates the user gaze point coordinates, and the motion control unit generates a three-dimensional space trajectory instruction according to the user instructions. The six-degree-of-freedom robotic arm performs the three-dimensional pose adjustment of the display screen to ensure that the rotation accuracy ≤ 0.1° and the translation positioning error < 1mm;

[0104] S4. The intelligent perception module captures the user's biometric features and environmental space data in real time, including millimeter-wave radar arrays for detecting moving objects, binocular stereo vision modules for three-dimensional reconstruction, surface electromyography sensors for capturing the user's upper limb movement intentions, and an adaptive control module that dynamically corrects the pitch and azimuth angles of the display screen based on the real-time data stream to keep the deviation between the screen normal vector and the user's visual axis direction less than 3°.

[0105] S5. When it is detected that the user performs a standing-sitting conversion, a height compensation algorithm is triggered to keep the height difference between the center point of the display screen and the user's eyes within ±50 mm. In the multi-person meeting mode, a periodic scanning protocol is started, and the display screen orientation is dynamically adjusted according to the sound source localization of the speaker.

[0106] S6. Use a ToF camera array to build an indoor point cloud model with a resolution of 512×512×256 voxels. The environmental semantic segmentation unit identifies the semantic labels of furniture and electrical appliances categories and establishes a dynamic passable area map. The user can mark multiple preferred stopping poses in the three-dimensional model through a custom interface.

[0107] S7. When the acceleration at the end of the robotic arm exceeds 4 m / s 2 Activate dynamic damping control. During the movement of the robot, if a sudden obstacle appears on the detected path, execute an emergency brake within a time ≤ 200 ms.

[0108] Embodiment 1

[0109] Adaptive adjustment in the home entertainment scenario

[0110] Scenario description: When the user moves from the sofa to the bar while watching a movie in the living room, the system tracks the user's position in real time and adjusts the pose of the display screen.

[0111] System operation:

[0112] 1. Path planning: Use lidar SLAM to build a 2D grid map (resolution 5 cm). When using the A* algorithm to plan the movement path, dynamically avoid pets (detection distance 2 m, response time 150 ms).

[0113] 2. Robotic arm control: The end effector of the six-axis robotic arm adjusts the display screen at an angular velocity of 0.05° / s. Based on the user's skeletal key points (shoulder coordinate error ±10 mm) identified by a spatio-temporal convolutional neural network (sampling rate 30 Hz), the Kalman filter is used to predict the user's position in the next 3 seconds.

[0114] 3. Visual compensation: The binocular vision module (baseline length 12 cm) calculates the coordinates of the user's pupil center, and uses the EPnP algorithm to solve the pitch angle of the display screen to make the angle between the screen normal and the line of sight ≤ 2.5°.

[0115] Technical indicators:

[0116] Robot moving speed 0.8m / s, emergency stop deceleration 4m / s 2

[0117] The repeatability of the robot arm joint is ±0.08°

[0118] Eye tracking delay ≤80ms.

[0119] Example 2

[0120] Dynamic response in multi-person conference mode

[0121] Scenario description: Four people in a conference room take turns speaking, and the display automatically turns to the current speaker.

[0122] System Operation:

[0123] 1. Sound source localization: Using a 64-channel microphone array (aperture 40cm), the GCC-PHAT algorithm is used to calculate the time difference of arrival (TDOA), with a positioning accuracy of ±5°.

[0124] 2. Multi-target tracking: Millimeter-wave radar (operating frequency 77 GHz) detects human outlines within 5 meters. The DBSCAN clustering algorithm distinguishes different users and associates sound sources with human body heat maps (update rate 10 Hz).

[0125] 3. Robotic arm coordination: The three-axis gimbal rotates at 90° / s, coordinated with the translation of the robotic arm base (speed 0.5m / s). The joint angles are calculated through the inverse Jacobian matrix to ensure that the end linear velocity is stable at 1.2m / s.

[0126] Technical indicators:

[0127] Speaker switching response time ≤ 1.2s

[0128] Display screen rotation accuracy ±1.5°

[0129] Voiceprint recognition accuracy ≥98%.

[0130] Example 3

[0131] Accurate 3D space reconstruction and obstacle avoidance

[0132] Scenario description: Automatically build navigation maps and avoid temporary obstacles in a complex office environment.

[0133] System Operation:

[0134] 1. 3D modeling: ToF camera (940nm wavelength) collects depth data at 30fps and generates voxel map (512×512×256, voxel size 2cm) through TSDF algorithm.3 )。

[0135] 2. Semantic segmentation: The PointNet++ network (parameter scale 1.2M) is used to classify obstacles in real time. The recognition rate of chairs is 92%, and the recognition rate of desks is 95%.

[0136] 3. Dynamic obstacle avoidance: When a suddenly emerging moving object (speed > 1m / s) is detected, the RRT* algorithm generates a new path within 200ms, and the expansion radius of obstacles in the cost map is set to 1.2 times the diameter of the robot.

[0137] Technical indicators:

[0138] Map update delay ≤ 50ms

[0139] Obstacle re - recognition rate ≥ 90%

[0140] Path planning success rate > 99.5%.

[0141] Example 4

[0142] Fine control in the medical rehabilitation scenario

[0143] Scenario description: Provide ergonomic display pose adjustment for wheelchair users.

[0144] System operation:

[0145] 1. Biometric recognition: The surface electromyography sensor (sampling rate 2kHz) captures the forearm sEMG signal, and decodes the user's gesture intention through the LSTM network (128 hidden units). The classification accuracy is 89%.

[0146] 2. Safety protection: The six - axis mechanical wrist six - dimensional force sensor (range ±50N / ±5Nm) detects the contact force. When the pressure exceeds 15N, admittance control (virtual stiffness 200N / m, damping coefficient 40Ns / m) is triggered.

[0147] 3. Adaptive adjustment: Combining nine - axis IMU data (noise density 12μg / √Hz) and depth camera information, the extended Kalman filter is used to fuse the user's pose data to keep the center height of the display ±30mm different from the user's eye height.

[0148] Technical indicators:

[0149] Gesture recognition delay ≤ 100ms

[0150] Contact force control accuracy ±0.8N

[0151] Height maintenance stability ±2mm / 10min.

[0152] Example 5

[0153] Medical surgical navigation scenario

[0154] Scenario description: During the operation, the surgeon needs to continuously look at the 3D organ model on the display screen, and the system tracks the micro-movements of the head in real time and compensates for the pose of the display screen

[0155] System operation:

[0156] 1. Eye tracking: Using the pupil-corneal reflection vector method (sampling rate 240Hz), blink noise is removed through the RANSAC algorithm, and the line-of-sight positioning accuracy reaches ±0.5°

[0157] 2. Dynamic compensation: When it is detected that the doctor's head translation > 2mm (nine-axis IMU data), the end of the robotic arm performs inverse kinematics solution with a 5ms delay to keep the angle between the screen normal and the line of sight < 1°

[0158] 3. Safety control: The six-axis force sensor on the wrist (resolution 0.01N) monitors the contact force of the robotic arm in real time, and activates the virtual fixture when the distance from the surgical equipment < 10cm

[0159] Technical indicators:

[0160] Line-of-sight tracking delay ≤ 15ms

[0161] Display screen pose compensation speed ≥ 50mm / s

[0162] The rendering frame rate of the three-dimensional model is synchronized to 120fps

[0163] Example 6

[0164] Industrial design review scenario

[0165] Scenario description: The designer observes the CAD model from multiple angles around the physical prototype, and the display screen automatically rotates around with the observation position. System operation:

[0166] 1. Multi-target tracking: The millimeter-wave radar (bandwidth 4GHz) detects the movement trajectory of personnel within 5m, and predicts the position in the next 2 seconds through Kalman filtering

[0167] 2. Cooperative movement: The six-degree-of-freedom robotic arm and the omnidirectional chassis are linked, and the inverse solution of the Jacobian matrix is used to achieve end trajectory tracking (maximum combined speed 1.5m / s)

[0168] 3. Visual enhancement: The ToF camera (depth accuracy ±1mm) reconstructs the physical prototype in real time, and aligns the virtual model with the physical object through the ICP algorithm

[0169] Technical indicators:

[0170] The error of the circumferential movement trajectory < 3mm

[0171] Virtual-real alignment accuracy: ±0.5mm

[0172] Multi-target tracking refresh rate: 60Hz

[0173] Example 7

[0174] Virtual reality fitness scenario

[0175] Scene description: When the user plays a motion-sensing game, the display screen adaptively adjusts the viewing angle according to the movement amplitude

[0176] System operation:

[0177] 1. EMG signal analysis: The surface electrodes (8 channels) collect the sEMG signals of the biceps brachii, and the SVM classifier (kernel function RBF) is used to identify the arm-swinging action, with an accuracy of 93%

[0178] 2. Motion prediction: The LSTM network (64 hidden units) predicts the displacement of the body's center of mass in the next 0.5 seconds based on 10 consecutive frames of skeletal data (30fps)

[0179] 3. Dynamic response: The three-axis gimbal completes the pitch adjustment of ±45° within 200ms, and coordinates with the chassis movement compensation for large displacement technical indicators:

[0180] Action recognition delay ≤ 50ms

[0181] Viewing angle matching error < 2°

[0182] Emergency braking distance < 100mm (when the user suddenly falls).

[0183] Example 8

[0184] Academic lecture hall scenario

[0185] Scene description: When the speaker moves, the display screen follows intelligently to ensure that the PPT content is always facing the best viewing angle of the audience

[0186] System operation:

[0187] 1. Hybrid positioning: The UWB base station (accuracy ±10cm) and the visual odometer are fused to locate the speaker's position, and the extended Kalman filter is used to reduce the cumulative error

[0188] 2. Group line-of-sight optimization: The binocular camera array constructs the audience heat map, and the genetic algorithm is used to calculate the optimal orientation of the display screen (the fitness function includes that 80% of the audience's line-of-sight angles < 15°)

[0189] 3. Smooth transition: The cubic spline interpolation is used to plan the motion trajectory of the robotic arm, and the joint angular acceleration is limited to 2rad / s 2 Technical indicators within:

[0190] Group line-of-sight coverage rate ≥ 85%

[0191] Trajectory smoothness (acceleration change rate) < 0.5 rad / s 3

[0192] Multi-source positioning data fusion frequency: 100 Hz

[0193] Comparative Example 1

[0194] Fixed eye tracking system

[0195] Traditional solution:

[0196] The display screen is fixed on the desktop, and only relies on a monocular camera (30 fps) to achieve line-of-sight tracking

[0197] The traditional threshold method is used to detect the pupil center, without head movement compensation

[0198] Performance defects:

[0199] 1. When the user's head moves laterally by 15 cm, the line-of-sight mapping error reaches ±8°

[0200] 2. Manual calibration is required every 5 minutes, interrupting the work process

[0201] 3. After continuous use for 1 hour, the neck muscle fatigue increases by 40% (EMG monitoring data).

[0202] Comparative Example 2

[0203] Manually adjust the display screen bracket

[0204] Traditional solution:

[0205] The mechanical hydraulic bracket supports three-degree-of-freedom adjustment

[0206] Relies on the user to manually operate the knob to adjust the pose

[0207] Performance defects:

[0208] 1. The average time-consuming for a single adjustment is 23 seconds (measured data)

[0209] 2. The repeat positioning accuracy is only ±5°, which cannot meet the requirements of fine operations

[0210] 3. It is not equipped with an environment perception module, and the collision accident rate reaches 1 time per 8 hours.

[0211] For specific comparison data, see Table 1:

[0212] Index Embodiment of the present invention Comparative Example 1 Comparative Example 2 Gaze tracking accuracy (°) ≤0.5 ±8 N / A Pose adjustment delay (ms) ≤50 300 23000 Increase in user fatigue (%) ＜5 40 25 Environmental adaptability Dynamic obstacle avoidance Fixed type Static

[0213] By comparing the examples with the comparative examples, it is proved that the present invention has obvious advantages in terms of dynamic compensation accuracy, response speed and ergonomics.

[0214] In addition, the components designed in the present invention are all common standard components or components known to those skilled in the art. Their structures and principles can all be learned by those skilled in the art through technical manuals or by conventional experimental methods. Those skilled in the art can fully implement them without further elaboration. The content protected by the present invention does not involve improvements to the internal structure and methods either.

[0215] The embodiments disclosed in the present invention are preferred embodiments, but are not limited thereto. Those of ordinary skill in the art can easily understand the spirit of the present invention based on the above embodiments and make different extensions and changes. However, as long as they do not depart from the spirit of the present invention, they are all within the protection scope of the present invention.

Claims

1. A dynamic display screen orientation adjustment system based on user behavior, characterized in that, Comprising: A mobile robot platform configured to achieve indoor autonomous movement through a bottom drive mechanism, the drive mechanism including a high-precision motion component; A six-degree-of-freedom robotic arm module vertically installed on the top of the robot platform, with a display installation interface configured at the end; An intelligent sensing module integrated with an infrared sensor array, a nine-axis inertial measurement unit, and a depth vision sensor, configured to capture user biometric features and environmental space data in real time; An adaptive control module including a multi-core processor and a motion control unit, the multi-core processor configured to run machine learning algorithms to analyze user body language features, and the motion control unit synchronously generating three-dimensional space trajectory instructions; Wherein, the robot platform is navigated to a user-specified area through the drive mechanism; The six-degree-of-freedom robotic arm is used to perform three-dimensional pose adjustment of the display screen, with a rotation accuracy ≤ 0.1° and a translational positioning error < 1 mm; Based on the real-time data stream of the intelligent sensing module, the pitch angle and azimuth angle of the display screen are dynamically corrected to keep the deviation between the screen normal vector and the user's visual axis direction < 3°.

2. The system for dynamically adjusting the display screen direction based on user behavior according to claim 1, characterized in that, The high-precision motion component adopts an omnidirectional wheel system and lidar SLAM combined navigation, and its path planning module is configured to: establish a cost map including dynamic obstacle prediction, and generate an optimal movement path through the A* algorithm; when it is detected that the user's line-of-sight focus deviation exceeds a preset threshold, trigger an autonomous obstacle avoidance replanning mechanism.

3. The dynamic display screen orientation adjustment system based on user behavior according to claim 1, wherein The six-degree-of-freedom robotic arm module includes: a series joint structure, each joint configured with a harmonic reducer and an absolute encoder; the end effector is provided with a three-axis pan-tilt mechanism, and its rotation range covers ±180° azimuth angle and ±90° pitch angle; a force / torque sensor is integrated at the wrist of the robotic arm to monitor external disturbances in real time and trigger active compliance control.

4. The dynamic display screen orientation adjustment system based on user behavior according to claim 1, wherein The intelligent sensing module further includes: a millimeter-wave radar array configured circumferentially on the robot platform for detecting moving objects within 5 meters; a binocular stereo vision module equipped with an infrared structured light projector to achieve a three-dimensional reconstruction accuracy of ±2 mm within a distance of 0.1 - 5 meters; a surface electromyogram sensor configured to capture user upper limb movement intention signals.

5. The system for dynamically adjusting the display screen direction based on user behavior according to claim 1, wherein The adaptive control module includes: a behavior recognition sub-module that processes continuous 20-frame skeletal key point data using a spatio-temporal convolutional neural network; a gaze tracking sub-module that calculates three-dimensional gaze point coordinates by combining corneal reflection vectors and pupil center coordinates; a dynamic weight fusion unit that performs Bayesian probability fusion on the confidence levels of head pose, gesture commands, and line-of-sight focus.

6. The dynamic display screen orientation adjustment system based on user behavior according to claim 1, characterized in that It further includes a multi-modal interaction interface, including: an augmented reality projection device that superimposes a virtual control panel within the user's field of view; a voice interaction unit that supports the intention recognition of natural language commands; a tactile feedback device integrated at the operation terminal of the robotic arm to provide operating force feedback guidance.

7. The dynamic display screen orientation adjustment system based on user behavior according to claim 1, characterized in that, It further includes an implementation scenario adaptive strategy: when it is detected that the user performs a standing-sitting conversion, trigger a height compensation algorithm to keep the height difference between the center point of the display screen and the user's eyes within ±50 mm; in a multi-person meeting mode, start a periodic scanning protocol to dynamically adjust the display screen orientation according to the sound source localization of the speaker.

8. The dynamic display screen orientation adjustment system based on user behavior according to claim 1, characterized in that It also includes a three-dimensional space modeling engine: using a ToF camera array to construct an indoor point cloud model with a resolution of 512×512×256 voxels; an environmental semantic segmentation unit that identifies semantic labels of furniture and electrical appliances categories and establishes a dynamic passable area map; a user-defined interface that supports marking multiple preferred parking poses in the three-dimensional model.

9. The dynamic display screen orientation adjustment system based on user behavior according to claim 1, characterized in that, It also includes a safety protection mechanism: when the acceleration at the end of the robotic arm exceeds 4 m / s 2 , dynamic damping control is activated; during the movement of the robot, if a sudden obstacle appears on the detected path, the emergency braking time ≤ 200 ms.

10. A method for dynamically adjusting the display screen direction based on user behavior, including the system for dynamically adjusting the display screen direction based on user behavior according to any one of claims 1-9, characterized in that, It includes the following steps: S1. Start the mobile robot platform, initialize the bottom drive mechanism and the high-precision motion components, initialize the six-degree-of-freedom robotic arm module, check the status of the harmonic reducers and absolute encoders of each joint, start the intelligent perception module, including an infrared sensor array, a nine-axis inertial measurement unit, a depth vision sensor, etc., ensure the normal real-time data stream, start the adaptive control module, load the machine learning algorithm by the multi-core processor, and the motion control unit is ready to receive instructions; S2. The user specifies the robot's moving area through a multi-modal interaction interface, such as an augmented reality projection device and a voice interaction unit. The adaptive control module receives the user's instructions, generates the optimal moving path through the path planning module, and the robot platform moves to the specified area according to the navigation instructions, using the omnidirectional wheel system and the lidar SLAM combined navigation; S3. The user indicates the desired pose of the display screen through a multi-modal interaction interface or body language, such as gestures and head postures. The behavior recognition sub-module of the adaptive control module processes the user's body language features, and the gaze tracking sub-module calculates the coordinates of the user's fixation point. The motion control unit generates a three-dimensional space trajectory instruction according to the user's instructions, and the six-degree-of-freedom robotic arm executes the three-dimensional pose adjustment of the display screen to ensure that the rotation accuracy is ≤0.1° and the translational positioning error is <1 mm; S4. The intelligent perception module captures the user's biometric and environmental space data in real time, including detecting moving objects by the millimeter-wave radar array, realizing three-dimensional reconstruction by the binocular stereo vision module, and capturing the user's upper limb movement intention by the surface electromyography sensor. The adaptive control module dynamically corrects the pitch angle and azimuth angle of the display screen according to the real-time data stream, keeping the deviation between the screen normal vector and the user's visual axis direction <3°; S5. When it is detected that the user performs a standing-sitting conversion, trigger the height compensation algorithm to keep the height difference between the center point of the display screen and the user's eyes within ±50 mm. In the multi-person meeting mode, start the periodic scanning protocol and dynamically adjust the orientation of the display screen according to the sound source localization of the speaker; S6. Use the ToF camera array to construct an indoor point cloud model with a resolution of 512×512×256 voxels. The environmental semantic segmentation unit identifies the semantic labels of furniture and electrical appliances categories and establishes a dynamic passable area map. The user can mark multiple preferred parking poses in the three-dimensional model through the custom interface; S7. When the acceleration at the end of the robotic arm exceeds 4 m / s 2 activate dynamic damping control. During the movement of the robot, if a sudden obstacle appears on the detected path, perform an emergency brake with a time ≤ 200 ms.

Images