Mobile terminal screen centering adjustment system driven by VR equipment and method thereof

Abstract

The invention discloses a mobile terminal screen centering adjustment system driven by VR equipment and a method thereof, and particularly relates to the technical field of VR equipment, the system comprises a multi-modal sensing module, a dual-drive execution mechanism, an intelligent control unit and a magnetic type quick-change interface, and the method specifically comprises the steps of S1, multi-modal data acquisition, S2, space coordinate transformation, S3, dynamic parameter generation and S4, cooperative adjustment execution. The multi-mode sensing module is matched with the intelligent control unit and the dual-drive executing mechanism, automatic adjustment is achieved, a user can conveniently use the glasses after wearing the glasses, meanwhile, high-precision adjustment is achieved on the basis of a specific algorithm technically, the screen position precision reaches + / -0.05 mm, the lens adjustment precision is high, clear vision is guaranteed, and the user experience is improved. And the magnetic type quick-change interface can be automatically identified to adapt to different mobile terminals, so that the system can adapt to different user habits, is widely applied to multiple fields such as entertainment, professional design and education and medical treatment, and promotes the application and development of the VR technology.

Description

Technical Field

[0001] The present invention relates to the technical field of VR devices. More specifically, the present invention relates to a system and method for adjusting the center of a mobile terminal screen driven by a VR device. Background Art

[0002] With the rapid development of virtual reality technology, VR devices have been widely used in many fields such as entertainment, education, medical treatment, and industrial design. In VR experiences, as a content presentation carrier, the center adjustment of the mobile terminal screen and its adaptation to user visual parameters are crucial, directly affecting the user's immersion and usage experience.

[0003] However, there are many problems with current VR devices in these aspects. For example: 1. The adjustment of the mobile terminal screen by traditional VR devices mainly relies on manual operations. The common methods are through adjustment knobs, buttons, or sliding tracks on the device. Users need to rely on their own visual judgment to manually adjust the position of the mobile terminal in the device to achieve screen centering. This manual adjustment method is not only cumbersome. For users who are using it for the first time or are not familiar with the device, it is difficult to complete the adjustment quickly and accurately. Moreover, during the user's use, if the head position changes significantly, the screen is likely to shift again, and at this time, the user needs to manually adjust it again, seriously affecting the coherence and immersion of use. For example, in some VR game scenarios, the intense head movement of players may cause the screen position to deviate, and frequently interrupting the game to perform manual adjustment greatly destroys the game experience. 2. In terms of adjustment accuracy, there are also obvious limitations in existing technical means. The manual adjustment method is limited by the visual error of the human eye and the physical accuracy of the adjustment mechanism, and it is very difficult to achieve precise adjustment at the millimeter level or even the sub-millimeter level. This results in the screen being unable to be accurately aligned with the user's pupil center, easily causing visual deviation, and further leading to the user's visual fatigue and dizziness. Especially for users who use VR devices for a long time, this discomfort caused by insufficient adjustment accuracy is more obvious, seriously restricting the usage duration and application scope of VR devices. 3. Existing VR devices also perform poorly in adapting to the interpupillary distance and focal length of different users. The interpupillary distance of different individuals varies, generally between 55 - 70 mm. At the same time, the eyesight conditions of users are different, and the requirements for the lens focal length are also different. However, traditional VR devices often only provide limited fixed adjustment gears and cannot perform personalized precise adaptation according to the specific facial features and visual parameters of users. This makes it difficult for some users to obtain clear and comfortable visual effects during use and unable to fully utilize the advantages of VR devices, to a certain extent hindering the popularization and promotion of VR technology.

[0004] In summary, existing VR devices have problems such as low automation, poor adjustment accuracy, and insufficient adaptability in the centering adjustment of the mobile terminal screen and the adaptation of user visual parameters. Therefore, the present invention provides a system and method for centering adjustment of a mobile terminal screen driven by a VR device. Summary of the Invention

[0005] To overcome the above-mentioned defects of the prior art, the present invention provides a system and method for centering adjustment of a mobile terminal screen driven by a VR device to solve the problems raised in the above background technology.

[0006] To achieve the above object, the present invention provides the following technical solution: A system for centering adjustment of a mobile terminal screen driven by a VR device, comprising: A multi-modal perception module; It includes a binocular camera and an infrared structured light sensor. Among them, the binocular camera is installed inside the VR device near the human eye area and is used to capture image information including the edges of the mobile terminal screen and the user's pupils in real time. By processing these images, the coordinates of the screen edge feature points and the position information of the pupils can be obtained. The infrared structured light sensor is arranged at the front end of the VR device and is used to scan the user's face to obtain three-dimensional geometric parameters of the face, such as interpupillary distance, nasal bridge height, and orbital contour data; A dual-drive actuator; It consists of a mobile phone clamping component driven by an X-Y biaxial linear motor and a magnetic levitation lens adjustment device. Among them, the X-Y biaxial linear motor is connected to the mobile phone clamping component and can drive the mobile phone clamping component to move precisely in the X-axis and Y-axis directions to adjust the position of the mobile terminal screen. The magnetic levitation lens adjustment device can adjust the lenses of the VR device, including adjusting the distance between the lenses and the distance between the lenses and the screen, so as to achieve the adaptation of interpupillary distance and focal length; An intelligent control unit; It is respectively connected to the multi-modal perception module and the dual-drive actuator. According to the calculated screen offset and lens adjustment parameters, the intelligent control unit generates corresponding drive instructions and sends them to the dual-drive actuator; The intelligent control unit also processes and analyzes the data from the binocular camera and the infrared structured light sensor based on a multi-modal data fusion algorithm. The algorithm includes: A pupil and screen coordinate mapping model based on a homography matrix, which can convert the two-dimensional coordinates of the screen and the pupils obtained from the image into three-dimensional space coordinates, and then accurately calculate the three-dimensional space position deviation of the screen center relative to the pupil center; A joint optimization strategy for lens spacing and focal length based on facial parameters. According to the obtained three-dimensional geometric parameters of the face, considering the relationship between interpupillary distance and focal length, the optimal lens adjustment parameters are generated, and the adjustment response time ≤ 50 milliseconds; Magnetic fast - change interface It is set in the mobile phone placement area of the VR device, integrating a pressure sensor and an NFC chip. The pressure sensor can detect the pressure distribution pattern when the mobile phone is placed. By analyzing the pressure distribution, the size and placement state of the mobile phone can be identified. The NFC chip can perform near - field communication with the mobile phone to read the device ID information stored in the mobile phone, so as to realize the automatic identification of the mobile phone model and load the corresponding preset parameters according to the identification result.

[0007] Preferably, the binocular camera adopts active infrared fill - light technology. In low - light environments, by emitting infrared light to illuminate the shooting area, clear images of the pupil and the screen edge can be obtained, ensuring the accurate extraction of feature points and coordinate calculation in the image.

[0008] Preferably, the dual - drive actuator includes: The X - Y axis guide rail of the mobile phone clamping component uses crossed roller bearings, which can provide high - precision linear motion, enabling the motion accuracy of the mobile phone clamping component to reach ≤0.05 mm, ensuring the accuracy of screen position adjustment; The lead - screw nut mechanism of the lens adjustment device has a self - locking function. When the lens is adjusted to the specified position, the lead - screw nut mechanism can prevent the lens from shifting in position due to external force or vibration. At the same time, it also supports dynamic load compensation and can automatically adjust the driving force according to the load change during the lens adjustment process, ensuring the stability and accuracy of the adjustment.

[0009] Preferably, the multi - modal data fusion algorithm of the intelligent control unit includes: Based on the spatio - temporal alignment processing of multi - modal data, the data collected by the binocular camera and the infrared structured - light sensor at different times and spaces are aligned and fused to ensure the consistency and accuracy of the data, providing a reliable data basis for subsequent calculations and analyses; Based on the prediction of lens adjustment parameters using the facial three - dimensional model, by analyzing and modeling the obtained facial three - dimensional geometric parameters, the lens spacing and focal length adjustment parameters suitable for the user are predicted, improving the efficiency and accuracy of the adjustment.

[0010] Preferably, the magnetic fast - change interface realizes mobile phone model identification in the following way: The pressure sensor array is distributed on the surface of the magnetic fast - change interface. When the mobile phone is placed on the interface, the pressure sensor array will detect the pressure of the mobile phone on each position, forming a pressure distribution pattern. By analyzing and comparing this pressure distribution pattern with the pre - stored database, the size and approximate type of the mobile phone can be identified; The NFC chip communicates with the mobile phone in a near - field manner, reads the device ID information stored in the mobile phone, combines the recognition result of the pressure sensor, accurately determines the model of the mobile phone, and loads the corresponding preset parameters from the storage module, such as screen resolution and display ratio.

[0011] The present invention also provides a method for adjusting the center of the mobile - terminal screen driven by a VR device. Using the above - mentioned system for adjusting the center of the mobile - terminal screen driven by a VR device, it specifically includes the following steps: S1. Modal data acquisition: Use a binocular camera to obtain the position information of the edge feature points of the mobile - terminal screen and the user's pupils. During the shooting process, if the ambient light is dim, the binocular camera will activate the active infrared fill - light technology to ensure clear images are obtained. At the same time, the infrared structured - light sensor scans the user's face to obtain three - dimensional geometric parameters of the face, such as the inter - eye distance and the height of the nose bridge. S2. Spatial coordinate transformation: Establish a screen coordinate system and a pupil coordinate system. By calibrating the coordinates of the four corner points of the screen edge in the world coordinate system, and based on the corresponding relationship between the pupil center and the screen corner points, solve the homography matrix. Use this homography matrix to convert the two - dimensional coordinates of the screen and the pupil in the image into three - dimensional space coordinates, and then calculate the three - dimensional offset of the screen center relative to the pupil center. S3. Dynamic parameter generation: According to the calculated three - dimensional offset, generate movement instructions for the mobile - phone clamping component in the X - axis and Y - axis directions to drive the mobile - phone clamping component to move to a suitable position. Combining the obtained face parameters, according to the combined optimization strategy of the lens spacing and focal length, calculate the adjustment parameters for the horizontal and vertical distances of the lenses. The adjustment range of the horizontal spacing is 55 - 70 millimeters, and the adjustment accuracy of the vertical distance can reach 0.01 millimeters. S4. Cooperative adjustment execution: After receiving the movement instructions, the linear motor adopts a micro - step drive mode to drive the mobile - phone clamping component to move with a control accuracy of 0.01 millimeters per pulse. The magnetic - levitation device realizes the closed - loop control of the lens position according to the lens adjustment parameters through current feedback, and synchronously adjusts the spacing of the lenses and the distance from the screen to achieve the dynamic alignment of the screen and the pupils.

[0012] Preferably, in step S2, the coordinate transformation matrix is established in the following way: S2.1. Calibrate the four corner points of the screen edge to determine their exact coordinates in the world coordinate system. S2.2. By analyzing the corresponding relationship between the pupil center and the screen corner points in the image, use mathematical methods to solve the homography matrix. This homography matrix can accurately convert the two - dimensional coordinates of the screen and the pupil in the image into three - dimensional space coordinates, so as to accurately calculate the screen offset.

[0013] Preferably, in step S3, the generation of lens adjustment parameters includes: S3.1. According to the facial interpupillary distance parameter, within the adjustment range of 55 - 70 millimeters, accurately calculate the lateral distance of the lens to adapt to the interpupillary distances of different users; S3.2. Based on the image sharpness evaluation function, evaluate and analyze the image sharpness at different longitudinal distances of the lens, thereby optimizing the longitudinal distance of the lens to make the adjustment accuracy reach 0.01 millimeter, ensuring that users obtain a clear visual experience.

[0014] Preferably, it further includes: A real-time compensation mechanism. By analyzing and learning the historical trajectory data of the user's head movement, a head movement trajectory prediction model is established. The head movement trajectory prediction model can predict the movement direction and amplitude of the user's head in advance, and adjust the position of the driving mechanism in advance according to the prediction result to timely compensate for the dynamic offset between the screen and the lens, ensuring that the screen is always aligned with the pupil; Optimization of the adjustment strategy. Real-time monitor the user's blink frequency and the change of the fixation point. When the user's blink frequency is high or the fixation point changes rapidly, it indicates that the user may be dissatisfied with the current visual effect. At this time, dynamically adjust the response priority of the control algorithm, speed up the adjustment speed or adjust the adjustment amplitude to improve the user's visual experience.

[0015] Preferably, in step S4, the collaborative adjustment execution includes: The linear motor adopts a microstep drive mode, further subdividing each pulse of the motor to make the control accuracy reach 0.01 millimeter / pulse, enabling precise movement of the mobile phone clamping component; The magnetic levitation device realizes closed-loop control of the lens position through current feedback, real-time monitors the actual position of the lens, compares it with the target position, and adjusts the current magnitude according to the comparison result, thereby precisely controlling the position of the lens to ensure the accuracy and stability of the lens adjustment.

[0016] The technical effects and advantages of the present invention: 1. The system of the present invention can automatically adjust the position of the mobile terminal screen and the VR lens parameters according to the user's facial features and pupil position. Without the need for cumbersome manual operations, users can quickly obtain a VR experience with the screen centered, clear vision, and adapted to their own interpupillary distance and focal length, greatly improving the convenience and comfort of use, reducing the dizziness caused by discomfort during wearing, and being particularly suitable for long-time immersive VR scenarios; 2. Through the multi-modal perception module combined with advanced algorithms, the present invention achieves a screen position adjustment accuracy of ±0.05 mm, a lens horizontal spacing adjustment accuracy of 0.018°, and a vertical adjustment accuracy of 0.01 mm, ensuring that the screen and the pupil are always accurately aligned, and the image is always clear and sharp. Compared with traditional VR devices, it realizes a qualitative leap in visual effects and lays a foundation for the high-quality presentation of VR content; 3. The magnetic adsorption type quick-change interface of the present invention combines a pressure sensor and an NFC chip, which can automatically identify multiple mobile phone models and load corresponding parameters, enabling this VR device to be compatible with mobile terminals of different brands and sizes, expanding the scope of application of the device. At the same time, the head movement trajectory prediction model and the adjustment strategy optimization mechanism enable the system to adapt to the dynamic usage habits of different users, and can provide stable and high-quality VR services in fields such as daily entertainment, education and learning, and professional design. Brief Description of the Drawings

[0017] Figure 1 It is the overall system diagram of the present invention.

[0018] Figure 2 It is the method flow diagram of the present invention.

[0019] The reference numerals are: 1. Multi-modal perception module; 2. Dual-drive actuator; 3. Intelligent control unit; 4. Magnetic adsorption type quick-change interface; 10. Binocular camera; 11. Infrared structured light sensor; 20. Mobile phone clamping assembly; 21. Magnetic levitation lens adjustment device; 40. Pressure sensor; 41. NFC chip. Detailed Embodiments

[0020] Next, the technical solutions in the embodiments of the present invention will be clearly and completely described in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are only a part of the embodiments of the present invention, rather than all the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those of ordinary skill in the art without creative efforts shall fall within the protection scope of the present invention.

[0021] Embodiment 1. As shown in the attached Figure 1 figure, the present invention provides a mobile terminal screen centering adjustment system driven by a VR device. The specific functions of each module of this system are as follows: The multi-modal perception module 1 includes a binocular camera 10 and an infrared structured light sensor 11; The binocular camera 10 is a miniature CMOS camera with a resolution of 1280×720 pixels and a frame rate of 60fps, ensuring that it can quickly and clearly capture the screen edge and pupil images. The camera lens adopts fisheye correction technology to reduce image distortion and ensure the accurate extraction of feature points in the image. It is precisely installed inside the VR device near the human eye area, about 20mm away from the human eye, and ensures that the deviation between the camera optical axis and the human eye line of sight direction is within ±1°. The camera is connected to the intelligent control unit 3 through a USB3.0 interface, and transmits the collected image data to the intelligent control unit 3 in real time. In the intelligent control unit 3, the Canny algorithm is used to extract the screen edge feature points, and the pupil position coordinates are determined by the method based on gray projection; The infrared structured light sensor 11 is a sensor module composed of an infrared structured light projector with an emission wavelength of 850nm and a supporting infrared camera, which can obtain facial three-dimensional geometric data with an accuracy of ±0.5mm within a working distance of 500mm. It is installed at the front end of the VR device, at the center position of the front of the VR device, ensuring that it can completely scan the user's face. The infrared structured light sensor 11 communicates with the intelligent control unit 3 through an SPI interface. When collecting data, the infrared projector projects a specific structured light pattern onto the user's face, and the infrared camera captures the reflected pattern. The intelligent control unit 3 uses the principle of triangulation and the phase unwrapping algorithm to calculate the three-dimensional geometric parameters of the user's face, such as the interpupillary distance, the height of the nose bridge, and the orbital contour data.

[0022] The dual-drive actuator 2 is composed of a mobile phone clamping component 20 driven by an X-Y biaxial linear motor and a magnetic levitation lens adjustment device 21; The mobile phone clamping component 20 adopts an X-Y biaxial guide rail made of aluminum alloy. The guide rail slider is connected to the mobile phone clamping frame by bolts, ensuring firm connection and stable movement. The mobile phone clamping frame adopts an adjustable design, which can adapt to mobile phones of different sizes, and increases the friction force through rubber pads to prevent the mobile phone from slipping. At the same time, an X-Y biaxial linear motor with a maximum thrust of 5N and a stroke of 50mm is selected. The motor is connected to the guide rail through a coupling. The motor is controlled by the intelligent control unit 3 through pulse signals, and the pulse frequency is 1000Hz. The precise movement of the motor is achieved by controlling the pulse quantity and direction. The motor drive chip adopts an L298N chip with overcurrent protection function to ensure the safe and reliable operation of the motor. The X-Y axis guide rail adopts crossed roller bearings, which can effectively reduce the clearance and vibration during the movement of the guide rail, making the movement accuracy of the mobile phone clamping component 20 reach ≤0.05 mm. At the same time, limit switches are installed at both ends of the guide rail to prevent the motor from over-running; The magnetic levitation lens adjustment device 21 is composed of two parallel magnetic plates on the upper and lower sides and a lens bracket in the middle. The lens bracket is connected to the magnetic plates through a flexible connection structure to ensure that the lens can move freely in the vertical direction. The lateral adjustment between the lenses is achieved through a lead screw-nut mechanism. The lead screw is connected to the motor through a synchronous belt. The lateral adjustment lead screw motor is a stepper motor with a step angle of 1.8°. The step angle is subdivided to 0.018° through a microstepping driver to achieve precise adjustment of the lateral distance between the lenses. The adjustment range is 55 - 70 mm. The magnetic levitation adjustment in the vertical direction is achieved by controlling the current magnitude of the magnetic plates. The relationship between the current and the lens position is adjusted through a PID control algorithm to ensure that the adjustment accuracy of the lens in the vertical direction reaches 0.01 mm. The lead screw-nut mechanism has a self-locking function. When the motor stops rotating, the lead screw-nut can maintain the current position to prevent the lens from shifting due to external force or vibration. At the same time, a load compensation algorithm is added to the motor control program to automatically adjust the output torque of the motor according to the load change during the lens adjustment process to ensure the stability and accuracy of the adjustment.

[0023] The intelligent control unit 3 uses an embedded processor based on the ARMCortex-A72 core with a main frequency of 2.4 GHz, equipped with 4GB DDR4 memory and 32GB eMMC storage. The processor integrates rich interfaces, including USB, SPI, and I2C, which are convenient for communicating with the multi-modal perception module 1 and the dual-drive actuator 2. The intelligent control unit 3 runs the Linux operating system. A multi-modal data fusion algorithm and a device control program are developed on top of the operating system. The algorithm and the program are written in the C++ language, using the OpenCV library for image processing and the Eigen library for matrix operations. The multi-modal data fusion algorithm of the intelligent control unit 3 is implemented as follows: Based on the homography matrix pupil-to-screen coordinate mapping model, during the system initialization phase, a calibration board is used to calibrate the four corner points of the screen edge to obtain their coordinates in the world coordinate system. During the operation, according to the corresponding relationship between the pupil center and the screen corner points in the image, the homography matrix is solved using the eight-point method. Through this matrix, the two-dimensional coordinates of the screen and the pupil in the image are converted into three-dimensional space coordinates, and then the three-dimensional space position deviation of the screen center relative to the pupil center is accurately calculated. Based on the facial parameter lens distance and focal length joint optimization strategy, according to the obtained three-dimensional facial geometric parameters, the lens distance and focal length adjustment parameters suitable for the user are predicted through a pre-trained neural network model. During the adjustment process, the Laplacian variance function is used to evaluate the image sharpness in real time, and the focal length parameters are dynamically adjusted to make the adjustment response time ≤ 50 milliseconds. Drive instruction generation and communication: Based on the calculated screen offset and lens adjustment parameters, the intelligent control unit 3 generates corresponding drive instructions. For the mobile phone clamping assembly 20, the number and direction of pulses that the motor needs to move are calculated according to the screen offset, and the instructions are sent to the motor drive module through the serial port. For the magnetic levitation lens adjustment device 21, a PWM control signal is generated according to the lens adjustment parameters and sent to the control chip through the SPI interface to achieve precise control of the lens position.

[0024] The magnetic adsorption type quick-change interface 4 uses neodymium iron boron permanent magnet as the magnetic adsorption material, which can provide sufficient adsorption force to ensure the firm installation of the mobile phone. The interface surface is equipped with a pressure sensor array 40 composed of 16 pressure sensors 40 evenly distributed. The NFC chip 41 selects the PN532 chip of NXP and is connected to the intelligent control unit 3 through the I2C interface. When the mobile phone is placed on the magnetic adsorption type quick-change interface 4, the pressure sensor array 40 detects the pressure of the mobile phone on each position to form a pressure distribution pattern. The intelligent control unit 3 compares this pressure distribution pattern with the pre-stored database to initially identify the size and approximate type of the mobile phone. At the same time, the NFC chip 41 communicates with the mobile phone in the near field to read the device ID information stored in the mobile phone. The intelligent control unit 3 combines the recognition results of the pressure sensor 40 and the device ID information read by the NFC to accurately determine the mobile phone model and load the corresponding preset parameters, such as screen resolution and display ratio, from the storage module for subsequent screen position adjustment and image display optimization.

[0025] Embodiment 2, as shown in the attached Figure 2 figure, the embodiment of the present invention provides a method for centering the screen of a mobile terminal driven by a VR device. Using the above-mentioned system for centering the screen of a mobile terminal driven by a VR device, the method specifically includes the following steps: S1. Multi-modal data acquisition: After the user wears the VR device, the binocular camera 10 is immediately activated. If the ambient light is dim, the camera automatically activates the active infrared fill light technology to emit infrared light with a wavelength of 850nm to illuminate the shooting area. The camera collects image information including the edges of the mobile terminal screen and the user's pupils in real time at a frame rate of 60fps and transmits the image data to the intelligent control unit 3 through the USB3.0 interface. While the binocular camera 10 collects images, the infrared structured light sensor 11 scans the user's face. The infrared projector projects a specific structured light pattern onto the user's face, and the infrared camera captures the reflected pattern. The sensor module transmits the collected image data to the intelligent control unit 3 through the SPI interface. The intelligent control unit 3 uses the principle of triangulation and the phase unwrapping algorithm to calculate the three-dimensional geometric parameters of the user's face, such as interpupillary distance and nasal bridge height data; S2. Spatial coordinate transformation. In the intelligent control unit 3, the lower left corner of the screen is defined as the origin of the screen coordinate system, the horizontal direction of the screen is the X-axis, and the vertical direction is the Y-axis. At the same time, the center of the user's left eye pupil is defined as the origin of the pupil coordinate system, the direction of the line connecting the left and right eye pupils is the X-axis, and the direction perpendicular to this line and upward is the Y-axis. In the system initialization stage, the four corner points of the screen edge are calibrated through a calibration board to obtain the accurate coordinates of the corner points in the world coordinate system. During the operation process, according to the corresponding relationship between the pupil center and the screen corner points in the image, the homography matrix is solved using the eight-point method. The specific steps are as follows: Extract the image coordinates of the four corner points of the screen edge and the image coordinates of the pupil center from the images collected by the binocular camera 10. Combine these image coordinates with the pre-calibrated world coordinates of the corner points to form corresponding point pairs, and use the eight-point method to solve the homography matrix. This matrix can convert the two-dimensional coordinates of the screen and the pupil in the image into three-dimensional space coordinates; Through the obtained homography matrix, convert the two-dimensional coordinates of the screen center in the image into three-dimensional coordinates in the world coordinate system, and at the same time convert the two-dimensional coordinates of the pupil center into three-dimensional coordinates. Then calculate the three-dimensional offset of the screen center relative to the pupil center, including the offsets in the X-axis direction, Y-axis direction, and Z-axis direction.

[0026] S3. Dynamic parameter generation. According to the calculated three-dimensional offset of the screen center relative to the pupil center, determine the distances that the mobile phone clamping component 20 needs to move in the X-axis and Y-axis directions. The intelligent control unit 3 calculates the number and direction of pulses that the motor needs to move based on the pulse equivalent of the motor of 0.01 mm / pulse, and generates the motion instructions for the mobile phone clamping component 20 in the X-axis and Y-axis directions; The lens adjustment parameters are calculated as follows: Horizontal spacing adjustment. According to the facial interpupillary distance parameter, within the adjustment range of 55 - 70 mm, accurately calculate the horizontal spacing of the lens using the linear interpolation algorithm to adapt to different user interpupillary distances; Vertical distance adjustment. Based on the image sharpness evaluation function, such as the Laplacian variance function, evaluate and analyze the image sharpness at different vertical distances of the lens. By continuously adjusting the vertical distance of the lens, make the image sharpness reach the maximum value, thereby optimizing the vertical distance of the lens to make the adjustment accuracy reach 0.01 mm; S4. Coordinated adjustment execution. After receiving the motion instruction sent by the intelligent control unit 3, the linear motor adopts the micro-step drive mode to further subdivide each pulse of the motor, so that the control accuracy reaches 0.01 mm / pulse. The motor drives the mobile phone clamping bracket on the X-Y biaxial guide rail through a coupling to move the mobile phone screen to a proper position. The magnetic levitation device realizes the closed-loop control of the lens position through current feedback according to the lens adjustment parameters sent by the intelligent control unit 3, monitors the actual position of the lens in real time, compares it with the target position, and adjusts the current magnitude according to the comparison result, so as to accurately control the lens spacing and the distance from the screen, and realize the dynamic alignment of the screen and the pupil. During the adjustment process, the horizontal adjustment lead screw motor drives the lead screw to rotate through a synchronous belt according to the calculated horizontal lens spacing, so as to realize the horizontal adjustment of the lens. The magnetic levitation adjustment in the vertical direction moves the lens in the vertical direction by controlling the current magnitude of the magnetic pole plate, so as to achieve the purpose of optimizing the focal length.

[0027] Embodiment 3: On the basis of Embodiment 2, this embodiment further includes a real-time compensation mechanism and an adjustment strategy optimization, which are specifically as follows: Real-time compensation mechanism. By analyzing and learning the historical trajectory data of the user's head movement, a head movement trajectory prediction model using the Kalman filtering algorithm is established. This model predicts the user's head movement direction and amplitude in real time. When it predicts that the user's head is about to move, it adjusts the position of the driving mechanism in advance according to the prediction result to timely compensate for the dynamic offset between the screen and the lens, and ensure that the screen is always aligned with the pupil; Adjustment strategy optimization. The user's blink frequency and fixation point changes are monitored in real time. The blink frequency is calculated by analyzing the user's eye state in the images collected by the binocular camera 10, and the user's fixation point position is determined by using the eye movement tracking algorithm. When the user's blink frequency is relatively high or the fixation point changes rapidly, it indicates that the user may be dissatisfied with the current visual effect. At this time, the response priority of the control algorithm is dynamically adjusted to speed up the adjustment speed or adjust the adjustment amplitude to improve the user's visual experience. For example, when the blink frequency exceeds 30 times per minute, the weight of the image clarity evaluation function is increased to speed up the focal length adjustment speed; when the fixation point moves more than a certain distance within a short time, the moving speed of the mobile phone clamping component 20 is increased to quickly adjust the screen position.

[0028] The above are only the preferred embodiments of the present invention and are not used to limit the present invention. Any modifications, equivalent replacements, improvements, etc. made within the spirit and principle of the present invention shall be included in the protection scope of the present invention.

Claims

1. A mobile terminal screen centering adjustment system driven by a VR device, characterized in that Including: A multi-modal perception module (1); It includes a binocular camera (10) and an infrared structured light sensor (11). Among them, the binocular camera (10) is installed inside the VR device near the human eye area, and is used to capture in real time the image information including the edge of the mobile terminal screen and the user's pupil. By processing these images, the coordinates of the feature points on the screen edge and the position information of the pupil can be obtained. The infrared structured light sensor (11) is arranged at the front end of the VR device and is used to scan the user's face to obtain three-dimensional geometric parameters of the face, such as interpupillary distance, nasal bridge height, and orbital contour data; A dual-drive actuator (2); It consists of a mobile phone clamping component (20) driven by an X-Y biaxial linear motor and a magnetic levitation lens adjustment device (21). Among them, the X-Y biaxial linear motor is connected to the mobile phone clamping component (20) and can drive the mobile phone clamping component (20) to move precisely in the X-axis and Y-axis directions to adjust the position of the mobile terminal screen. The magnetic levitation lens adjustment device (21) can adjust the lenses of the VR device, including adjusting the distance between the lenses and the distance between the lenses and the screen, so as to achieve the adaptation of interpupillary distance and focal length; An intelligent control unit (3); It is respectively connected to the multi-modal perception module (1) and the dual-drive actuator (2). According to the calculated screen offset and lens adjustment parameters, the intelligent control unit (3) generates corresponding drive instructions and sends them to the dual-drive actuator (2); The intelligent control unit (3) also processes and analyzes the data from the binocular camera (10) and the infrared structured light sensor (11) based on a multi-modal data fusion algorithm. The algorithm includes: A pupil and screen coordinate mapping model based on the homography matrix, which can convert the two-dimensional coordinates of the screen and the pupil obtained from the image into three-dimensional space coordinates, and then accurately calculate the three-dimensional space position deviation of the screen center relative to the pupil center; A joint optimization strategy for lens spacing and focal length based on facial parameters. According to the obtained three-dimensional geometric parameters of the face, considering the relationship between interpupillary distance and focal length, the optimal lens adjustment parameters are generated, and the adjustment response time ≤ 50 milliseconds; A magnetic fast-changing interface (4); It is set in the mobile phone placement area of the VR device and integrates a pressure sensor (40) and an NFC chip (41). The pressure sensor (40) can detect the pressure distribution mode when the mobile phone is placed. By analyzing the pressure distribution, the size and placement state of the mobile phone can be identified. The NFC chip (41) can communicate with the mobile phone in the near field to read the device ID information stored in the mobile phone, so as to realize the automatic identification of the mobile phone model and load the corresponding preset parameters according to the identification result.

2. The mobile terminal screen centering adjustment system driven by a VR device according to claim 1, characterized in that: The binocular camera (10) adopts an active infrared fill light technology. In a low-light environment, by emitting infrared light to illuminate the shooting area, clear images of the pupil and the screen edge can be obtained, ensuring the accurate extraction and coordinate calculation of feature points in the image.

3. The mobile terminal screen centering adjustment system driven by the VR device according to claim 2, characterized in that: The dual-drive actuator (2) includes: The X-Y axis guide rail of the mobile phone clamping assembly (20) uses crossed roller bearings, which can provide high-precision linear motion, enabling the motion accuracy of the mobile phone clamping assembly (20) to reach ≤0.05 mm, ensuring the accuracy of screen position adjustment; The lead screw nut mechanism of the lens adjustment device has a self-locking function. When the lens is adjusted to the specified position, the lead screw nut mechanism can prevent the lens from shifting in position due to external force or vibration. At the same time, it also supports dynamic load compensation and can automatically adjust the driving force according to the load change during the lens adjustment process to ensure the stability and accuracy of the adjustment.

4. The mobile terminal screen centering adjustment system driven by a VR device according to claim 3, characterized in that: The multi-modal data fusion algorithm of the intelligent control unit (3) includes: Based on the spatio-temporal alignment processing of multi-modal data, the data collected by the binocular camera (10) and the infrared structured light sensor (11) at different times and spaces are aligned and fused to ensure the consistency and accuracy of the data, providing a reliable data basis for subsequent calculations and analyses; Based on the prediction of lens adjustment parameters based on the facial three-dimensional model, by analyzing and modeling the obtained facial three-dimensional geometric parameters, the lens spacing and focal length adjustment parameters suitable for the user are predicted, improving the efficiency and accuracy of the adjustment.

5. The mobile terminal screen centering adjustment system driven by a VR device according to claim 4, characterized in that: The magnetic fast-changing interface (4) realizes the identification of the mobile phone model in the following way: The pressure sensor (40) array is distributed on the surface of the magnetic fast-changing interface (4). When the mobile phone is placed on the interface, the pressure sensor (40) array will detect the pressure of the mobile phone on each position, forming a pressure distribution pattern. By analyzing and comparing this pressure distribution pattern with the pre-stored database, the size and approximate type of the mobile phone can be identified; The NFC chip communicates with the mobile phone in the near field, reads the device ID information stored in the mobile phone, and combines the recognition result of the pressure sensor (40) to accurately determine the model of the mobile phone and load the corresponding preset parameters from the storage module, such as the screen resolution and display ratio.

6. Method for adjusting the center of the mobile terminal screen driven by the VR device, using the system for adjusting the center of the mobile terminal screen driven by the VR device described in claim 5, characterized in that: Specifically, it includes the following steps: S1. Multi-modal data acquisition: Use the binocular camera (10) to obtain the position information of the edge feature points of the mobile terminal screen and the user's pupils. During the shooting process, if the ambient light is dim, the binocular camera (10) will activate the active infrared fill light technology to ensure clear images are obtained. At the same time, the infrared structured light sensor (11) scans the user's face to obtain the three-dimensional geometric parameters of the face, such as the interpupillary distance and the nasal bridge height; S2. Spatial coordinate transformation: Establish a screen coordinate system and a pupil coordinate system. By calibrating the coordinates of the four corner points of the screen edge in the world coordinate system and based on the corresponding relationship between the pupil center and the screen corner points, the homography matrix is solved. Using this homography matrix, the two-dimensional coordinates of the screen and the pupil in the image are transformed into three-dimensional space coordinates, and then the three-dimensional offset of the screen center relative to the pupil center is calculated; S3. Generate dynamic parameters. Based on the calculated three-dimensional offset, generate movement instructions for the mobile phone clamping component (20) in the X-axis and Y-axis directions to drive the mobile phone clamping component (20) to move to a suitable position. Combining the obtained facial parameters, according to the joint optimization strategy of lens spacing and focal length, calculate the adjustment parameters for the horizontal spacing and vertical distance of the lens. The adjustment range of the horizontal spacing is 55 - 70 mm, and the adjustment accuracy of the vertical distance can reach 0.01 mm. S4. Execute coordinated adjustment. After receiving the movement instruction, the linear motor adopts a micro-step drive mode and drives the mobile phone clamping component (20) to move with a control accuracy of 0.01 mm / pulse. The magnetic levitation device realizes the closed-loop control of the lens position according to the lens adjustment parameters, synchronously adjusts the spacing between the lenses and the distance from the screen, and realizes the dynamic alignment of the screen and the pupil.

7. The method according to claim 6, wherein: In step S2, the coordinate transformation matrix is established in the following way: S2.

1. Calibrate the four corner points of the screen edge to determine their exact coordinates in the world coordinate system. S2.

2. By analyzing the corresponding relationship between the pupil center and the screen corner points in the image, use mathematical methods to solve the homography matrix. This homography matrix can accurately convert the two-dimensional coordinates of the screen and the pupil in the image into three-dimensional space coordinates, thereby realizing the accurate calculation of the screen offset.

8. The method according to claim 6, wherein: In step S3, the generation of lens adjustment parameters includes: S3.

1. According to the facial interpupillary distance parameter, accurately calculate the horizontal spacing of the lens within the adjustment range of 55 - 70 mm to adapt to the interpupillary distances of different users. S3.

2. Based on the image sharpness evaluation function, evaluate and analyze the image sharpness at different longitudinal distances of the lens, thereby optimizing the longitudinal distance of the lens to make the adjustment accuracy reach 0.01 mm and ensure that users obtain a clear visual experience.

9. The method according to claim 6, characterized in that: It also includes: A real-time compensation mechanism. By analyzing and learning the historical trajectory data of the user's head movement, establish a head movement trajectory prediction model. This head movement trajectory prediction model can predict the movement direction and amplitude of the user's head in advance, and adjust the position of the drive mechanism in advance according to the prediction results to timely compensate for the dynamic offset between the screen and the lens, ensuring that the screen always remains aligned with the pupil. Optimization of the adjustment strategy. Real-time monitor the user's blink frequency and the change of the fixation point. When the user's blink frequency is high or the fixation point changes rapidly, it indicates that the user may be dissatisfied with the current visual effect. At this time, dynamically adjust the response priority of the control algorithm, accelerate the adjustment speed or adjust the adjustment amplitude to improve the user's visual experience.

10. The method according to claim 6, wherein: In step S4, the execution of coordinated adjustment includes: The linear motor adopts a micro-step drive mode, further subdivides each pulse of the motor, so that the control accuracy reaches 0.01 mm / pulse, and can realize the accurate movement of the mobile phone clamping component (20). The magnetic levitation device realizes the closed-loop control of the lens position through current feedback, real-time monitors the actual position of the lens, compares it with the target position, and adjusts the current magnitude according to the comparison result, thereby accurately controlling the position of the lens and ensuring the accuracy and stability of the lens adjustment.

Images