A dual-axis parallel visual tracking method and device

Abstract

This invention relates to a dual-axis parallel visual tracking method and device, comprising: acquiring an image of an occupant's face and identifying the two-dimensional pixel coordinates of the eyes and the tip of the nose in the occupant's face image; calculating the target sway angle of the screen based on the two-dimensional pixel coordinates; calculating the motion trajectory of the screen through the target sway angle; controlling the motion stroke of the actuator motor according to the running trajectory; automatically acquiring the user's gaze and the outline of the nose tip through real-time visual perception, achieving adaptive intelligent tracking without manual intervention, solving the problems of start-stop jitter and discontinuous motion in existing technologies that use simple point-to-point motion; generating a continuous and smooth motion curve through online trajectory planning, making the screen rotation smooth and natural, significantly improving visual comfort; this invention adopts three-closed-loop servo control, combined with real-time feedback and feedforward compensation, which can track complex trajectories with high precision and ensure stable system operation.

Description

Technical Field

[0001] This invention relates to the field of computer technology, and in particular to a dual-axis parallel visual tracking method, a dual-axis parallel visual tracking device, a computer device, and a storage medium. Background Technology

[0002] The current trend in in-vehicle screens is shifting from fixed displays to proactive adjustments based on user needs. The core challenge at this technological stage has moved from "whether it can physically rotate" to "how to rotate intelligently, automatically, and smoothly." The purpose of this invention is to enhance the intelligence of the control software at the existing physical structure level.

[0003] Currently, the mainstream software control scheme for achieving electric yaw of in-vehicle screens is an open-loop position control system based on preset commands. This involves manual operation of physical buttons, virtual buttons on the touchscreen, or sending simple voice commands, generating discrete trigger signals as system input. The current electric adjustment scheme for in-vehicle screens typically uses a software control logic based on a fixed preset open-loop program. Specifically, the control unit internally stores fixed angle parameters (e.g., yaw angle +15°, pitch angle 0°) corresponding one-to-one with different operating commands (e.g., "driver mode," "passenger mode"). Upon receiving a command, the core process executed by the system is: parsing the command → looking up the preset angle in a table → driving the motor to perform point-to-point movement. This motion typically relies on simple trapezoidal speed planning for start-stop control, and completes the process by rotating the screen to a preset position via PWM or CAN bus signals. This control mode is frequently seen in existing patents; for example, patent CN114313011A primarily relates to mechanical drive devices, and its underlying control logic falls into this category. CN117739076A focuses on mechanical structure design for smooth transmission, and its control core also follows the aforementioned mode. The software essence of this solution is a fixed-program system lacking external environment perception and real-time decision-making capabilities. While this mainstream control scheme achieves basic electric regulation, its internal software still has shortcomings, specifically:

[0004] (1) Lack of environmental awareness, the system is in a state of "blind control": The system does not have the ability to perceive the environment in real time, and all its behaviors depend on pre-programmed instructions. The software is "unaware" of the real usage scenario when it is running, which is the primary reason why it cannot achieve intelligence.

[0005] (2) Discontinuous motion experience: The motion trajectory is based on simple point-to-point planning, such as the trapezoidal speed curve used in patent CN117739076A, which may cause significant jitter during the start and stop phases, resulting in screen shaking. Simultaneously, due to the inability to respond to continuously changing input signals, the screen motion is not smooth, and steady-state jitter is large, severely affecting visual comfort. Summary of the Invention

[0006] In view of the above problems, embodiments of the present invention are proposed to provide a dual-axis parallel visual tracking method, a dual-axis parallel visual tracking device, a computer device, and a storage medium that overcome or at least partially solve the above problems.

[0007] To achieve the above objectives, embodiments of the present invention propose a dual-axis parallel visual tracking method, the method comprising:

[0008] The occupant's facial image is acquired, and the two-dimensional pixel coordinates of the eyes and the tip of the nose in the occupant's facial image are identified.

[0009] The target yaw angle of the screen is calculated based on the two-dimensional pixel coordinates.

[0010] The motion trajectory of the screen is calculated by the target yaw angle;

[0011] The motion stroke of the actuator motor is controlled according to the described running trajectory.

[0012] Preferably, the step of acquiring the occupant's facial image and identifying the two-dimensional pixel coordinates of the eyes and the tip of the nose in the occupant's facial image includes:

[0013] The occupant's facial image is normalized to obtain the horizontal coordinate, vertical coordinate, and relative depth. The normalized coordinates of the facial key points are then constructed using the horizontal coordinate, vertical coordinate, and relative depth.

[0014] The image pixel coordinates of the target key points are obtained by extracting them using the index of the normalized coordinates of the facial key points.

[0015] Preferably, calculating the target yaw angle of the screen based on the two-dimensional pixel coordinates includes:

[0016] The actual interpupillary distance, the pupil pixel distance in the image, and the camera focal length are obtained from the occupant's facial image. The three-dimensional coordinates of the eyes and the tip of the nose in the screen coordinate system are calculated using the camera intrinsic parameter matrix through the actual interpupillary distance, the pupil pixel distance in the image, and the camera focal length.

[0017] The yaw and pitch angles of the screen required to align with the line of sight are calculated using geometric relationships, and the yaw and pitch angles are combined to form the target yaw angle.

[0018] Preferably, the step of calculating the screen's motion trajectory through the target yaw angle includes:

[0019] The trajectory coefficients are obtained analytically based on the given initial and target positions and velocity boundary conditions.

[0020] Using the target yaw angle as input, the motion trajectory is generated by cubic polynomial spline interpolation based on the trajectory coefficients, and the first stroke and second stroke of the two actuators are calculated.

[0021] Preferably, controlling the motion stroke of the actuator motor according to the running trajectory includes:

[0022] Calculate the coordinates of the connection points between the two actuators and the screen in the base coordinate system;

[0023] Find the first stroke and the second first stroke of the two lead screws;

[0024] Calculate the Jacobian matrix to map the joint space angular velocity to the linear velocity of the motor actuator, and obtain the desired position and velocity.

[0025] The generated desired position and speed are used as inputs to the position loop. Based on the deviation between the desired angle and the actual angle feedback from the magnetic encoder, the motor speed command is output through PID control combined with integral limiting and speed feedforward.

[0026] The actuator motor's movement stroke is controlled according to the motor speed command.

[0027] This invention provides a dual-axis parallel visual tracking device, the device comprising:

[0028] The image data acquisition module is used to acquire occupant facial images and identify the two-dimensional pixel coordinates of the eyes and the tip of the nose in the occupant facial images;

[0029] The yaw angle calculation module is used to calculate the target yaw angle of the screen based on the two-dimensional pixel coordinates.

[0030] The motion trajectory calculation module is used to calculate the motion trajectory of the screen based on the target sway angle.

[0031] The control module is used to control the movement stroke of the actuator motor according to the running trajectory.

[0032] Preferably, the image data acquisition module includes:

[0033] The normalization processing submodule is used to perform normalization processing on the occupant's facial image to obtain the horizontal coordinate, vertical coordinate, and relative depth, and to construct the normalized coordinates of the facial key points through the horizontal coordinate, vertical coordinate, and relative depth.

[0034] The extraction submodule is used to extract the image pixel coordinates of the target key points by using the index of the normalized coordinates of the facial key points.

[0035] Preferably, the yaw angle calculation module includes:

[0036] The three-dimensional coordinate submodule is used to obtain the actual pupil distance, pupil pixel distance in the image, and camera focal length in the occupant's facial image. Using the camera intrinsic parameter matrix, the three-dimensional coordinates of the eyes and nose tip in the screen coordinate system are calculated based on the actual pupil distance, pupil pixel distance in the image, and camera focal length.

[0037] The calculation submodule is used to calculate the yaw angle and pitch angle of the screen required for alignment with the line of sight through geometric relationships, and to combine the yaw angle and pitch angle to form the target yaw angle.

[0038] This invention discloses a computer device, including a memory and a processor. The memory stores a computer program, and the processor executes the computer program to implement the steps of the above-described dual-axis parallel visual tracking method.

[0039] This invention discloses a computer-readable storage medium storing a computer program thereon, which, when executed by a processor, implements the steps of the above-described dual-axis parallel visual tracking method.

[0040] In this embodiment of the invention, the user's gaze and nose tip contour are automatically acquired through real-time visual perception, achieving fully autonomous and adaptive intelligent tracking without human intervention. This solves the problems of start-stop jitter and discontinuous movement inherent in existing technologies that use simple point-to-point motion. The invention generates continuous and smooth motion curves through online trajectory planning, making screen rotation smooth and natural, significantly improving visual comfort. The invention employs a three-closed-loop FOC servo control, combined with real-time feedback and feedforward compensation, enabling high-precision tracking of complex trajectories and effectively suppressing internal and external interference such as vehicle vibration, ensuring stable system operation. It provides a complete "perception-decision-execution" software closed-loop architecture with clear modules, suitable not only for vehicle screens but also easily ported to other application scenarios requiring intelligent visual tracking and posture adjustment, offering greater versatility and scalability. Attached Figure Description

[0041] To more clearly illustrate the technical solutions in the embodiments of the present invention, the accompanying drawings used in the description of the embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0042] Figure 1 This is a hardware schematic diagram of a dual-axis parallel vision tracking system according to an embodiment of the present invention;

[0043] Figure 2 This is a planar schematic diagram of a dual-axis parallel vision tracking system according to an embodiment of the present invention;

[0044] Figure 3This is a flowchart illustrating the steps of an embodiment of a dual-axis parallel visual tracking method according to the present invention.

[0045] Figure 4 This is a structural block diagram of an embodiment of a dual-axis parallel vision tracking device according to an embodiment of the present invention;

[0046] Figure 5 This is an internal structural diagram of a computer device according to one embodiment. Detailed Implementation

[0047] To make the technical problems, technical solutions, and beneficial effects solved by the embodiments of the present invention clearer, the embodiments of the present invention will be further described in detail below with reference to the accompanying drawings and examples. It should be understood that the specific embodiments described herein are merely illustrative of the present invention and are not intended to limit the present invention.

[0048] Reference Figure 1 The diagram shows a hardware schematic of a dual-axis parallel vision tracking system according to an embodiment of the present invention, including a camera, screen, SOC, MCU, limit switch, magnetic encoder, lead screw, motor, etc. The embodiments of the present invention do not impose too many restrictions on the types of hardware components.

[0049] Reference Figure 2 The diagram shows a planar schematic of a dual-axis parallel vision tracking system according to an embodiment of the present invention, as shown below. Figure 2 As shown, the rotation angle of the screen can be controlled by two lead screws. In addition, the lead screw structure can be replaced by linear motors, gear rack mechanisms, rope / belt drive mechanisms, etc. The embodiments of the present invention do not impose too many restrictions on this. Specifically, linear motors can directly generate linear thrust without intermediate conversion mechanisms, have fast response and high precision, and are suitable for high-speed and high-precision scenarios. The motor drives the gear to rotate, and the meshing rack generates linear displacement. The structure is simple and has high transmission rigidity. The motor winds up and unwinds the wire rope or synchronous belt, and linear motion is achieved by the pulley block. The structure is lightweight and has low noise.

[0050] Reference Figure 3 The diagram illustrates a flowchart of an embodiment of a dual-axis parallel visual tracking method according to the present invention, which may specifically include the following steps:

[0051] Step 101: Obtain the occupant's facial image and identify the two-dimensional pixel coordinates of the eyes and the tip of the nose in the occupant's facial image;

[0052] Step 102: Calculate the target yaw angle of the screen based on the two-dimensional pixel coordinates;

[0053] Step 103: Calculate the motion trajectory of the screen by the target yaw angle;

[0054] Step 104: Control the motion stroke of the actuator motor according to the running trajectory.

[0055] In this embodiment of the invention, acquiring the occupant's facial image and identifying the two-dimensional pixel coordinates of the eyes and the tip of the nose in the occupant's facial image includes:

[0056] The occupant's facial image is normalized to obtain the horizontal coordinate, vertical coordinate, and relative depth. The normalized coordinates of the facial key points are then constructed using the horizontal coordinate, vertical coordinate, and relative depth.

[0057] The image pixel coordinates of the target key points are obtained by extracting them using the index of the normalized coordinates of the facial key points.

[0058] Further applied to embodiments of the present invention, the step of calculating the target yaw angle of the screen based on the two-dimensional pixel coordinates includes:

[0059] The actual interpupillary distance, the pupil pixel distance in the image, and the camera focal length are obtained from the occupant's facial image. The three-dimensional coordinates of the eyes and the tip of the nose in the screen coordinate system are calculated using the camera intrinsic parameter matrix through the actual interpupillary distance, the pupil pixel distance in the image, and the camera focal length.

[0060] The yaw and pitch angles of the screen required to align with the line of sight are calculated using geometric relationships, and the yaw and pitch angles are combined to form the target yaw angle.

[0061] In this embodiment of the invention, the step of calculating the motion trajectory of the screen by the target yaw angle includes:

[0062] The trajectory coefficients are obtained analytically based on the given initial and target positions and velocity boundary conditions.

[0063] Using the target yaw angle as input, the motion trajectory is generated by cubic polynomial spline interpolation based on the trajectory coefficients, and the first stroke and second stroke of the two actuators are calculated.

[0064] Further applied to embodiments of the present invention, the step of controlling the motion stroke of the actuator motor according to the running trajectory includes:

[0065] Calculate the coordinates of the connection points between the two actuators and the screen in the base coordinate system;

[0066] Find the first stroke and the second first stroke of the two lead screws;

[0067] Calculate the Jacobian matrix to map the joint space angular velocity to the linear velocity of the motor actuator, and obtain the desired position and velocity.

[0068] The generated desired position and speed are used as inputs to the position loop. Based on the deviation between the desired angle and the actual angle feedback from the magnetic encoder, the motor speed command is output through PID control combined with integral limiting and speed feedforward.

[0069] The actuator motor's movement stroke is controlled according to the motor speed command.

[0070] In this embodiment of the invention, occupant facial images are continuously acquired using an in-vehicle monocular camera. First, the camera intrinsic parameter matrix K and distortion coefficients are calibrated using the Zhang Zhengyou calibration method, and the rigid body transformation matrix (rotation matrix R and translation vector t) from the camera coordinate system to the screen coordinate system is determined. Then, the AI ​​model "MediaPipe" deployed on an embedded platform is used to perform real-time inference on the images, detecting the two-dimensional pixel coordinates (u, v) of key points such as the eyes and the tip of the nose.

[0071] Load and initialize the MediaPipe Face Mesh model. This model is a lightweight facial landmark detection network containing 468 3D facial landmarks. During initialization, set up real-time video stream processing, limit detection to a single face, and enable more accurate landmark detection (such as pupils, nose tip, etc.).

[0072] A single frame of BGR format image is captured from a monocular camera and converted to RGB format (since the MediaPipe model uses RGB input). The image size can be adjusted as needed to match the model input requirements (e.g., 640×480) to reduce computational cost. The preprocessed image is then input into the Face Mesh model for inference. The model output contains normalized coordinates (x, y, z) of facial keypoints, where x and y are in the range [0, 1], and z represents relative depth. The image pixel coordinates of the target keypoints are extracted using indices provided by the model (e.g., left pupil index 468, right pupil index 473, nose tip index 1). and .

[0073] Because the camera exhibits radial and tangential distortion, distortion correction is required using a pre-calibrated camera intrinsic parameter matrix K and distortion coefficients. The OpenCV function cv2.undistortPoints() is used to correct the pixel coordinates, resulting in distortion-free normalized planar coordinates.

[0074] The corrected normalized planar coordinates are back-projected to the pixel coordinate system through the intrinsic parameter matrix:

[0075] (1)

[0076] This refers to the intrinsic parameter matrix;

[0077] Finally, the precise two-dimensional pixel coordinates (u, v) are obtained;

[0078] Subsequently, using the actual interpupillary distance Pupil pixel distance in the image and camera focal length Calculate depth (z) using similar triangles:

[0079] (2)

[0080] Actual interpupillary distance is expressed as The pixel distance of the pupil in the image is represented as And the camera focal length is expressed as ;

[0081] Then, using the inverse of the camera intrinsic parameter matrix K, the three-dimensional coordinates (x, y, z) of the eyeball in the screen coordinate system are calculated:

[0082] (3)

[0083] in, This refers to the inverse of the intrinsic parameter matrix;

[0084] Finally, the required screen target yaw angle α and pitch angle β for alignment with the line of sight are calculated using geometric relationships.

[0085] (4)

[0086] in, It is the angle at which the screen tilts up and down; It refers to the angle of the screen tilting left and right;

[0087] Target deflection angle received from the output As input, firstly, a cubic polynomial spline interpolation algorithm is used to calculate the current screen pose. With target attitude Between these parameters, a smooth motion trajectory with continuous position, velocity, and acceleration is planned online. The trajectory is described by the following cubic polynomial:

[0088] (5)

[0089] Among them, coefficient Given the initial and target positions and velocity boundary conditions (such as initial / terminal velocities which can be set to preset values), The analysis yielded the following:

[0090] (6)

[0091] in, It is the current screen angle; It is the target screen angle; It is the current screen angular velocity; It is the current screen angular velocity; T is the position loop period, 10ms;

[0092] Based on the target attitude angle output by the trajectory planning module. The stroke of the two lead screws is calculated in real time using the analytical inverse kinematics model of the dual-axis parallel mechanism. and .

[0093] First, calculate the coordinates of the two lead screws, i.e., the two actuators, at connection points 2 and 4 with the screen in the base coordinate system:

[0094] (7)

[0095] in, It is the horizontal distance from the center of rotation to the drive arm; It is the length of the drive arm; It is the target attitude angle on the screen output by the trajectory planning module;

[0096] because Figure 2 The distances from points 2 and 4 to points 5 and 6 on the lead screw are the lengths of the constraint link. Given that points 5 and 6 are fixed and move only along the x-axis, while their values ​​along the y-axis and z-axis remain unchanged, determine the stroke of the two lead screws. and From the formula for the distance between two points, we can obtain:

[0097] (8)

[0098] in, It is the distance between screen point 2.4 and lead screw point 5.6; x1-x6, y1-y6, z1-z6 are the coordinates of the screen hinge point in the screen coordinate system;

[0099] This model, based on the geometric constraints of the mechanism, aims to achieve precise control of a dual-axis parallel mechanism. The system calculates the Jacobian matrix J in real time based on the geometric model, and then calculates the joint space angular velocity. Mapped to the linear velocity of the motor actuator :

[0100] (9)

[0101] The matrix calculation incorporates singular configuration judgment and avoidance logic, ensuring that all motion commands remain within the mechanism's workspace, thus guaranteeing the safety and naturalness of the motion.

[0102] The position loop is the outermost control loop of the entire servo control system, responsible for achieving high-precision position tracking. Its core task is to convert the desired position generated by the trajectory planning module into motor speed commands, and to ensure that the actual position accurately tracks the desired position through feedback control.

[0103] The desired location generated in real time and its speed As input to the position loop (20Hz), the current position of the two motors is read via two magnetic encoders on the left and right sides. The result is obtained by subtracting the desired position from the actual position fed back by the magnetic encoder.

[0104] (10)

[0105] By combining PID control with integral limiting and speed feedforward, the motor speed command is output:

[0106] (11)

[0107] in, It is the gain coefficient defined by the PID controller;

[0108] This instruction, after being transformed by the inverse Jacobi, serves as the reference for the innermost Field-Oriented Control (FOC) current loop. The current loop decouples the motor phase currents through Clarke / Park transformation to achieve fast direct torque control. The specific steps include:

[0109] The speed loop (100Hz) is located between the position loop (outer layer) and the current loop (inner layer). It is mainly responsible for converting the speed command output by the position loop into a current command, and ensuring that the actual speed accurately tracks the command through the internal PI controller.

[0110] The speed loop receives the speed reference value output by the position loop. To suppress the impact of sudden command changes on the system, the velocity reference value is first ramped up. The ramp generator operates based on a preset acceleration limit value. This ensures that the change in the output speed command within each control cycle does not exceed this limit, smoothly transitioning the step-like speed command into a continuously changing speed command. The actual mechanical angular velocity of the motor rotor can be obtained in real time through a magnetic encoder or photoelectric encoder. This serves as the feedback quantity for the speed loop. The error between the smoothed speed command and the feedback speed is calculated:

[0111] (12)

[0112] The error is handled using a proportional-integral (PI) controller.

[0113] (13)

[0114] in, This is the initial control input from the speed loop; The speed loop proportionality coefficient determines the system's response speed to speed errors; This is the velocity loop integral coefficient, used to eliminate steady-state velocity errors.

[0115] The current loop (1kHz) employs a field-oriented control (FOC) strategy, receiving q-axis current commands from the velocity loop. The three-phase current of the motor is measured using a current sensor. , , .

[0116] The Clarke transformation is then performed to convert the three-phase currents into two-phase currents in a stationary coordinate system.

[0117] (14)

[0118] Perform the Park transformation to convert the current in the stationary coordinate system into the current in the rotating dq coordinate system:

[0119] (15)

[0120] This gives us the current. and Output using the speed loop As the target current , usually set =0 achieves maximum torque control, then PI control continues.

[0121] (16)

[0122] (17)

[0123] Perform the inverse Park transformation to convert the voltage in the rotating coordinate system to the voltage in the stationary coordinate system:

[0124] (18)

[0125] Finally, the stationary coordinate system voltage command will be transferred. and The signal is converted into a three-phase PWM duty cycle signal. The generated three-phase PWM signal directly controls the switching state of the three-phase inverter, and finally generates the desired voltage waveform at the motor end.

[0126] A servo system drives a dual-axis mechanism to perform motion, changing the actual orientation of the screen. This new screen-environment geometry then becomes the input for the next visual perception. Any subtle change in the occupant's position triggers immediate trajectory replanning, driving the screen to smoothly follow the path again. Thus, visual perception, intelligent planning, and precise control form a real-time, dynamic closed loop, enabling the screen to continuously, automatically, and flexibly align with the occupant's line of sight, achieving truly seamless adaptive interaction.

[0127] In this embodiment of the invention, a dual-axis parallel vision tracking system is provided, comprising:

[0128] The visual perception module is used to output the user's eye coordinates in real time in the screen coordinate system based on the images acquired by the monocular camera, through AI facial key point detection and monocular 3D reconstruction algorithms.

[0129] The trajectory planning module, connected to the visual perception module, is used to calculate the screen target attitude angle based on the three-dimensional coordinates of the eyeball, and to generate a continuous and smooth screen motion trajectory with position, velocity, and acceleration using an online interpolation algorithm.

[0130] The servo control module, connected to the trajectory planning module, employs a cascaded three-loop magnetic field orientation control system consisting of a position loop, a velocity loop, and a current loop to drive the movement of the dual-axis parallel mechanism, aligning the screen normal with the user's line of sight.

[0131] (2) Perception level: The monocular 3D reconstruction algorithm is based on pre-calibrated camera intrinsic parameters and camera-screen coordinate system transformation relationship. The visual perception module also includes a filtering unit for multi-frame smoothing filtering of the output eye coordinates or pose angles. The AI ​​face key point detection model is specifically the MediaPipe Face Mesh model.

[0132] (3) Decision-making level: The online interpolation algorithm is a cubic polynomial spline interpolation algorithm. The trajectory planning module is also used to execute based on the Jacobian matrix of the dual-axis parallel mechanism.

[0133] (4) Execution level: In the servo control module, the position loop controller is a composite PID controller combined with speed feedforward. The execution cycles of the current loop, speed loop, and position loop are 50µs, 1ms, and 10ms, respectively. The current loop of the servo control module adopts field-oriented control based on Clarke transform and Park transform, and drives the motor through space vector pulse width modulation.

[0134] In this embodiment of the invention, the user's gaze is automatically acquired through real-time visual perception, achieving fully autonomous and adaptive intelligent tracking without human intervention. Existing technologies use simple point-to-point motion, which suffers from start-stop jitter and discontinuous motion. This invention generates continuous and smooth motion curves through online trajectory planning, making screen rotation smooth and natural, significantly improving visual comfort. This invention adopts a three-closed-loop FOC servo control, combined with real-time feedback and feedforward compensation, which can accurately track complex trajectories and effectively suppress internal and external interference such as vehicle vibration, ensuring stable system operation. It provides a complete "perception-decision-execution" software closed-loop architecture with clear modules. It is not only suitable for vehicle screens but also easy to port to other application scenarios that require intelligent visual tracking and attitude adjustment, making it more versatile and scalable.

[0135] It should be noted that, for the sake of simplicity, the method embodiments are all described as a series of actions. However, those skilled in the art should understand that the embodiments of the present invention are not limited to the described order of actions, because according to the embodiments of the present invention, some steps can be performed in other orders or simultaneously. Furthermore, those skilled in the art should also understand that the embodiments described in the specification are preferred embodiments, and the actions involved are not necessarily essential to the embodiments of the present invention.

[0136] Reference Figure 4 The diagram illustrates a structural block diagram of an embodiment of a dual-axis parallel visual tracking device according to the present invention, which may specifically include the following modules:

[0137] The image data acquisition module 301 is used to acquire the occupant's facial image and identify the two-dimensional pixel coordinates of the eyes and the tip of the nose in the occupant's facial image;

[0138] The yaw angle calculation module 302 is used to calculate the target yaw angle of the screen based on the two-dimensional pixel coordinates.

[0139] The motion trajectory calculation module 303 is used to calculate the motion trajectory of the screen by the target sway angle.

[0140] The control module 304 is used to control the motion stroke of the actuator motor according to the running trajectory.

[0141] Preferably, the image data acquisition module includes:

[0142] The normalization processing submodule is used to perform normalization processing on the occupant's facial image to obtain the horizontal coordinate, vertical coordinate, and relative depth, and to construct the normalized coordinates of the facial key points through the horizontal coordinate, vertical coordinate, and relative depth.

[0143] The extraction submodule is used to extract the image pixel coordinates of the target key points by using the index of the normalized coordinates of the facial key points.

[0144] Preferably, the yaw angle calculation module includes:

[0145] The three-dimensional coordinate submodule is used to obtain the actual pupil distance, pupil pixel distance in the image, and camera focal length in the occupant's facial image. Using the camera intrinsic parameter matrix, the three-dimensional coordinates of the eyes and nose tip in the screen coordinate system are calculated based on the actual pupil distance, pupil pixel distance in the image, and camera focal length.

[0146] The calculation submodule is used to calculate the yaw angle and pitch angle of the screen required for alignment with the line of sight through geometric relationships, and to combine the yaw angle and pitch angle to form the target yaw angle.

[0147] Preferably, the motion trajectory calculation module includes:

[0148] The trajectory coefficient submodule is used to analytically obtain trajectory coefficients from given initial and target position and velocity boundary conditions;

[0149] The polynomial spline interpolation processing submodule is used to generate a motion trajectory by performing cubic polynomial spline interpolation based on the trajectory coefficients, using the target yaw angle as input, and to calculate the first stroke and second first stroke of the two actuators.

[0150] Preferably, the control module includes:

[0151] The coordinate calculation submodule is used to calculate the coordinates of the connection points between the two actuators and the screen in the base coordinate system;

[0152] The solver submodule is used to solve for the first stroke and the second first stroke of the two lead screws;

[0153] The calculation submodule is used to calculate the Jacobian matrix, which maps the joint space angular velocity to the linear velocity of the motor actuator, and obtains the desired position and velocity.

[0154] The motor speed command submodule generates the desired position and speed as input to the position loop. Based on the deviation between the desired angle and the actual angle feedback from the magnetic encoder, it outputs the motor speed command through PID control combined with integral limiting and speed feedforward.

[0155] The control submodule is used to control the movement stroke of the actuator motor according to the motor speed command.

[0156] Each module in the aforementioned dual-axis parallel vision tracking device can be implemented entirely or partially through software, hardware, or a combination thereof. These modules can be embedded in the processor of a computer device in hardware form or independent of it, or stored in the memory of a computer device in software form, so that the processor can call and execute the corresponding operations of each module.

[0157] The dual-axis parallel vision tracking device provided above can be used to execute the dual-axis parallel vision tracking method provided in any of the above embodiments, and has corresponding functions and beneficial effects.

[0158] In one embodiment, a computer device is provided, which may be an automotive electronic instrument panel device, and its internal structure diagram may be as follows: Figure 5 As shown, the automotive electronic instrument panel includes a processor, memory, network interface, display screen, and input devices connected via a system bus. The processor provides computing and control capabilities. The memory includes non-volatile storage media and internal memory. The non-volatile storage media stores the operating system and computer programs. The internal memory provides an environment for the operation of the operating system and computer programs stored in the non-volatile storage media. The network interface is used to communicate with external terminals via a network connection. When the computer program is executed by the processor, it implements a dual-axis parallel visual tracking method. The display screen can be an LCD screen or an e-ink screen. The input devices can be a touch layer covering the display screen, buttons, a trackball, or a touchpad mounted on the computer device casing, or an external keyboard, touchpad, or mouse.

[0159] Those skilled in the art will understand that Figure 5 The structure shown is merely a block diagram of a portion of the structure related to the present application and does not constitute a limitation on the computer device to which the present application is applied. Specific computer devices may include more or fewer components than those shown in the figure, or combine certain components, or have different component arrangements.

[0160] In one embodiment, a computer device is provided, including a memory and a processor, wherein the memory stores a computer program, and the processor executes the computer program to perform the following steps:

[0161] The occupant's facial image is acquired, and the two-dimensional pixel coordinates of the eyes and the tip of the nose in the occupant's facial image are identified.

[0162] The target yaw angle of the screen is calculated based on the two-dimensional pixel coordinates.

[0163] The motion trajectory of the screen is calculated by the target yaw angle;

[0164] The motion stroke of the actuator motor is controlled according to the described running trajectory.

[0165] Preferably, the step of acquiring the occupant's facial image and identifying the two-dimensional pixel coordinates of the eyes and the tip of the nose in the occupant's facial image includes:

[0166] The occupant's facial image is normalized to obtain the horizontal coordinate, vertical coordinate, and relative depth. The normalized coordinates of the facial key points are then constructed using the horizontal coordinate, vertical coordinate, and relative depth.

[0167] The image pixel coordinates of the target key points are obtained by extracting them using the index of the normalized coordinates of the facial key points.

[0168] Preferably, calculating the target yaw angle of the screen based on the two-dimensional pixel coordinates includes:

[0169] The actual interpupillary distance, the pupil pixel distance in the image, and the camera focal length are obtained from the occupant's facial image. The three-dimensional coordinates of the eyes and the tip of the nose in the screen coordinate system are calculated using the camera intrinsic parameter matrix through the actual interpupillary distance, the pupil pixel distance in the image, and the camera focal length.

[0170] The yaw and pitch angles of the screen required to align with the line of sight are calculated using geometric relationships, and the yaw and pitch angles are combined to form the target yaw angle.

[0171] Preferably, the step of calculating the screen's motion trajectory through the target yaw angle includes:

[0172] The trajectory coefficients are obtained analytically based on the given initial and target positions and velocity boundary conditions.

[0173] Using the target yaw angle as input, the motion trajectory is generated by cubic polynomial spline interpolation based on the trajectory coefficients, and the first stroke and second stroke of the two actuators are calculated.

[0174] Preferably, controlling the motion stroke of the actuator motor according to the running trajectory includes:

[0175] Calculate the coordinates of the connection points between the two actuators and the screen in the base coordinate system;

[0176] Find the first stroke and the second first stroke of the two lead screws;

[0177] Calculate the Jacobian matrix to map the joint space angular velocity to the linear velocity of the motor actuator, and obtain the desired position and velocity.

[0178] The generated desired position and speed are used as inputs to the position loop. Based on the deviation between the desired angle and the actual angle feedback from the magnetic encoder, the motor speed command is output through PID control combined with integral limiting and speed feedforward.

[0179] The actuator motor's movement stroke is controlled according to the motor speed command.

[0180] In one embodiment, a computer-readable storage medium is provided having a computer program stored thereon, the computer program performing the following steps when executed by a processor:

[0181] The occupant's facial image is acquired, and the two-dimensional pixel coordinates of the eyes and the tip of the nose in the occupant's facial image are identified.

[0182] The target yaw angle of the screen is calculated based on the two-dimensional pixel coordinates.

[0183] The motion trajectory of the screen is calculated by the target yaw angle;

[0184] The motion stroke of the actuator motor is controlled according to the described running trajectory.

[0185] Preferably, the step of acquiring the occupant's facial image and identifying the two-dimensional pixel coordinates of the eyes and the tip of the nose in the occupant's facial image includes:

[0186] The occupant's facial image is normalized to obtain the horizontal coordinate, vertical coordinate, and relative depth. The normalized coordinates of the facial key points are then constructed using the horizontal coordinate, vertical coordinate, and relative depth.

[0187] The image pixel coordinates of the target key points are obtained by extracting them using the index of the normalized coordinates of the facial key points.

[0188] Preferably, calculating the target yaw angle of the screen based on the two-dimensional pixel coordinates includes:

[0189] The actual interpupillary distance, the pupil pixel distance in the image, and the camera focal length are obtained from the occupant's facial image. The three-dimensional coordinates of the eyes and the tip of the nose in the screen coordinate system are calculated using the camera intrinsic parameter matrix through the actual interpupillary distance, the pupil pixel distance in the image, and the camera focal length.

[0190] The yaw and pitch angles of the screen required to align with the line of sight are calculated using geometric relationships, and the yaw and pitch angles are combined to form the target yaw angle.

[0191] Preferably, the step of calculating the screen's motion trajectory through the target yaw angle includes:

[0192] The trajectory coefficients are obtained analytically based on the given initial and target positions and velocity boundary conditions.

[0193] Using the target yaw angle as input, the motion trajectory is generated by cubic polynomial spline interpolation based on the trajectory coefficients, and the first stroke and second stroke of the two actuators are calculated.

[0194] Preferably, controlling the motion stroke of the actuator motor according to the running trajectory includes:

[0195] Calculate the coordinates of the connection points between the two actuators and the screen in the base coordinate system;

[0196] Find the first stroke and the second first stroke of the two lead screws;

[0197] Calculate the Jacobian matrix to map the joint space angular velocity to the linear velocity of the motor actuator, and obtain the desired position and velocity.

[0198] The generated desired position and speed are used as inputs to the position loop. Based on the deviation between the desired angle and the actual angle feedback from the magnetic encoder, the motor speed command is output through PID control combined with integral limiting and speed feedforward.

[0199] The actuator motor's movement stroke is controlled according to the motor speed command.

[0200] The various embodiments in this specification are described in a progressive manner, with each embodiment focusing on the differences from other embodiments. The same or similar parts between the various embodiments can be referred to each other.

[0201] Those skilled in the art will understand that embodiments of the present invention can be provided as methods, apparatus, or computer program products. Therefore, embodiments of the present invention can take the form of entirely hardware embodiments, entirely software embodiments, or embodiments combining software and hardware aspects. Furthermore, embodiments of the present invention can take the form of computer program products implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0202] Embodiments of the present invention are described with reference to flowchart illustrations and / or block diagrams of methods, terminal devices (systems), and computer program products according to embodiments of the invention. It will be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, embedded processor, or other programmable data processing terminal device to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing terminal device, generate instructions for implementing the flowchart illustrations and / or block diagrams. Figure 1 One or more processes and / or boxes Figure 1 A device that provides the functions specified in one or more boxes.

[0203] These computer program instructions may also be stored in a computer-readable storage medium that can direct a computer or other programmable data processing terminal device to operate in a particular manner, such that the instructions stored in the computer-readable storage medium produce an article of manufacture including instruction means, which are implemented in a process Figure 1 One or more processes and / or boxes Figure 1 The function specified in one or more boxes.

[0204] These computer program instructions can also be loaded onto a computer or other programmable data processing terminal equipment, causing a series of operational steps to be performed on the computer or other programmable terminal equipment to produce a computer-implemented process, thereby providing instructions that execute on the computer or other programmable terminal equipment for implementing the process. Figure 1 One or more processes and / or boxes Figure 1 The steps of the function specified in one or more boxes.

[0205] Although preferred embodiments of the present invention have been described, those skilled in the art, upon learning the basic inventive concept, can make other changes and modifications to these embodiments. Therefore, the appended claims are intended to be interpreted as including the preferred embodiments as well as all changes and modifications falling within the scope of the embodiments of the present invention.

[0206] Finally, it should be noted that in this document, relational terms such as "first" and "second" are used only to distinguish one entity or operation from another, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Furthermore, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or terminal device that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or terminal device. Without further limitations, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or terminal device that includes said element.

[0207] The foregoing has provided a detailed description of a dual-axis parallel visual tracking method, a dual-axis parallel visual tracking device, a computer device, and a storage medium provided by the present invention. Specific examples have been used to illustrate the principles and implementation methods of the present invention. The descriptions of the above embodiments are only for the purpose of helping to understand the method and core ideas of the present invention. At the same time, for those skilled in the art, there will be changes in the specific implementation methods and application scope based on the ideas of the present invention. Therefore, the content of this specification should not be construed as a limitation of the present invention.

Claims

1. A dual-axis parallel visual tracking method, characterized in that, The method includes: The occupant's facial image is acquired, and the two-dimensional pixel coordinates of the eyes and the tip of the nose in the occupant's facial image are identified. The target yaw angle of the screen is calculated based on the two-dimensional pixel coordinates. The motion trajectory of the screen is calculated by the target yaw angle; The motion stroke of the actuator motor is controlled according to the described running trajectory.

2. The dual-axis parallel visual tracking method according to claim 1, characterized in that, The process of acquiring the occupant's facial image and identifying the two-dimensional pixel coordinates of the eyes and the tip of the nose in the occupant's facial image includes: The occupant's facial image is normalized to obtain the horizontal coordinate, vertical coordinate, and relative depth. The normalized coordinates of the facial key points are then constructed using the horizontal coordinate, vertical coordinate, and relative depth. The image pixel coordinates of the target key points are obtained by extracting them using the index of the normalized coordinates of the facial key points.

3. The dual-axis parallel visual tracking method according to claim 2, characterized in that, The step of calculating the target yaw angle of the screen based on the two-dimensional pixel coordinates includes: The actual interpupillary distance, the pupil pixel distance in the image, and the camera focal length are obtained from the occupant's facial image. The three-dimensional coordinates of the eyes and the tip of the nose in the screen coordinate system are calculated using the camera intrinsic parameter matrix through the actual interpupillary distance, the pupil pixel distance in the image, and the camera focal length. The yaw and pitch angles of the screen required to align with the line of sight are calculated using geometric relationships, and the yaw and pitch angles are combined to form the target yaw angle.

4. The dual-axis parallel visual tracking method according to claim 1, characterized in that, The process of calculating the screen's motion trajectory using the target yaw angle includes: The trajectory coefficients are obtained analytically based on the given initial and target positions and velocity boundary conditions. Using the target yaw angle as input, the motion trajectory is generated by cubic polynomial spline interpolation based on the trajectory coefficients, and the first stroke and second stroke of the two actuators are calculated.

5. The dual-axis parallel visual tracking method according to claim 1, characterized in that, The step of controlling the motion stroke of the actuator motor according to the running trajectory includes: Calculate the coordinates of the connection points between the two actuators and the screen in the base coordinate system; Find the first stroke and the second first stroke of the two lead screws; Calculate the Jacobian matrix to map the joint space angular velocity to the linear velocity of the motor actuator, and obtain the desired position and velocity. The generated desired position and speed are used as inputs to the position loop. Based on the deviation between the desired angle and the actual angle feedback from the magnetic encoder, the motor speed command is output through PID control combined with integral limiting and speed feedforward. The actuator motor's movement stroke is controlled according to the motor speed command.

6. A dual-axis parallel visual tracking device, characterized in that, The device includes: The image data acquisition module is used to acquire occupant facial images and identify the two-dimensional pixel coordinates of the eyes and the tip of the nose in the occupant facial images; The yaw angle calculation module is used to calculate the target yaw angle of the screen based on the two-dimensional pixel coordinates. The motion trajectory calculation module is used to calculate the motion trajectory of the screen based on the target sway angle. The control module is used to control the movement stroke of the actuator motor according to the running trajectory.

7. The dual-axis parallel visual tracking device according to claim 6, characterized in that, The image data acquisition module includes: The normalization processing submodule is used to perform normalization processing on the occupant's facial image to obtain the horizontal coordinate, vertical coordinate, and relative depth, and to construct the normalized coordinates of the facial key points through the horizontal coordinate, vertical coordinate, and relative depth. The extraction submodule is used to extract the image pixel coordinates of the target key points by using the index of the normalized coordinates of the facial key points.

8. The dual-axis parallel visual tracking device according to claim 7, characterized in that, The yaw angle calculation module includes: The three-dimensional coordinate submodule is used to obtain the actual pupil distance, pupil pixel distance in the image, and camera focal length in the occupant's facial image. Using the camera intrinsic parameter matrix, the three-dimensional coordinates of the eyes and nose tip in the screen coordinate system are calculated based on the actual pupil distance, pupil pixel distance in the image, and camera focal length. The calculation submodule is used to calculate the yaw angle and pitch angle of the screen required for alignment with the line of sight through geometric relationships, and to combine the yaw angle and pitch angle to form the target yaw angle.

9. A computer device comprising a memory and a processor, wherein the memory stores a computer program, characterized in that, When the processor executes the computer program, it implements the steps of the dual-axis parallel visual tracking method according to any one of claims 1 to 5.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by the processor, it implements the steps of the dual-axis parallel visual tracking method according to any one of claims 1 to 5.

Images