Method and device for determining pose of display screen control device, and storage medium

Abstract

Disclosed in the present application are a method and device for determining the pose of a display screen control device, and a storage medium. The method comprises: receiving the angle of a control device collected by a control device sensor, and acquiring the angle of a display screen by means of a display screen sensor; calculating the angle difference between the angle of the control device and the angle of the display screen; and identifying positioning point coordinates of the control device by means of a positioning sensor, and determining pose information of the control device on the basis of the angle difference and the positioning point coordinates.

Description

Methods, equipment and storage media for determining the pose of display screen control devices

[0001] This application claims priority to Chinese patent application No. 202510113940.X, filed on January 24, 2025, the entire contents of which are incorporated herein by reference. Technical Field

[0002] This application relates to the field of spatial positioning technology, and in particular to a method, device and storage medium for determining the pose of a display screen control device. Background Technology

[0003] To enable interaction between users and virtual scenes, VR displays typically use infrared cameras to capture the spatial coordinates of multiple positioning points in the control pen, thus forming the pen's pose in space.

[0004] In related technologies, to prevent the pose of the control pen from shifting during the tracking and positioning process, the VR display can use angle sensors such as gyroscopes and geomagnetic sensors in the control pen to collect angle information and calibrate the pose of the control pen.

[0005] However, to calibrate the pen's pose using the angle sensor, the VR display typically needs to determine the pen's pose based on the spatial information of the display's current location when receiving angle information from the angle sensor in the pen. Therefore, when the VR display's pose changes, the calibration result for the pen's pose obtained by the VR display after acquiring angle information through an infrared camera located on the VR display deviates from the actual result. This leads to insufficient accuracy in the pen's pose calibration.

[0006] The above content is only used to help understand the technical solution of this application and does not represent an admission that the above content is prior art. Technical issues

[0007] The main objective of this application is to provide a method, device, and storage medium for determining the pose of a display control device, aiming to solve the technical problem of insufficient accuracy of the pose calibration results of the control pen obtained by the angle sensor in the control pen of a VR display. Technical solutions

[0008] To achieve the above objectives, this application provides a method for determining the pose of a display screen control device, the method comprising the following steps:

[0009] The angle of the control device is acquired by the sensor of the control device, and the angle of the display screen is acquired by the sensor of the display screen;

[0010] Calculate the angle difference between the control device angle and the display screen angle;

[0011] The positioning sensors identify the coordinates of the positioning points of the control device, and the position and pose information of the control device is determined based on the angle difference and the coordinates of the positioning points.

[0012] In one embodiment, after the step of identifying the coordinates of the positioning point of the control device through the positioning sensor and determining the pose information of the control device based on the angle difference and the coordinates of the positioning point, the method further includes:

[0013] Receive control signals from the control device;

[0014] Based on the pose information and the control signal, the control command of the control device is determined;

[0015] Determine the interactive actions corresponding to the control commands in the virtual scene;

[0016] The interactive action is executed, and the interactive action and its result in the virtual scene are displayed on the screen.

[0017] In one embodiment, after the step of identifying the coordinates of the positioning point of the control device through the positioning sensor and determining the pose information of the control device based on the angle difference and the coordinates of the positioning point, the method further includes:

[0018] Based on the relative pose of the control device and the display screen, determine the extension line of the control device;

[0019] Determine the target extension line of the extension line in the virtual scene, and use the target extension line as the control line of the virtual scene;

[0020] The control line is displayed in the virtual scene shown on the display screen.

[0021] In one embodiment, the step of identifying the coordinates of the positioning point of the control device using a positioning sensor, and determining the pose information of the control device based on the angle difference and the coordinates of the positioning point, includes:

[0022] The positioning signal of the control device is received through the positioning sensor;

[0023] The coordinates of the control device's positioning point in the display screen coordinate system are determined based on the positioning signal.

[0024] Based on the coordinates of the positioning point and the angle difference, the position and pose information of the control device in the coordinate system of the display screen is determined.

[0025] In one embodiment, the step of determining the coordinates of the control device's positioning point in the display screen coordinate system based on the positioning signal includes:

[0026] The positioning image of the control device is acquired using at least two optical sensors;

[0027] Identify the positioning recognition light spot of the control device based on the positioning point in the positioning image;

[0028] The coordinates of the positioning point are determined by triangulation based on the position of the positioning spot mapped onto the positioning image and the sensor coordinates of the light sensor in the display screen coordinate system.

[0029] In one embodiment, the step of determining the coordinates of the control device's positioning point in the display screen coordinate system based on the positioning signal includes:

[0030] Based on image processing, the relative position of the control device and the display screen is determined, and the image acquired by the display screen control system is converted to grayscale to obtain a grayscale image.

[0031] The image after grayscale processing is binarized to obtain a binarized image;

[0032] Feature point detection is performed on the target object in the binarized image, feature points are extracted, and feature vectors are formed to determine the coordinates of the target in space.

[0033] In one embodiment, the step of determining the coordinates of the control device's positioning point in the display screen coordinate system based on the positioning signal includes:

[0034] The display screen control system uses ultra-wideband signals to determine the signal transmission time from when the control device sends the ultra-wideband signal to when the display screen receives the ultra-wideband signal, thereby determining the distance between the ultra-wideband radar and the ultra-wideband transmission position of the control device, and thus determining the positioning point coordinates of the control device.

[0035] In one embodiment, the step of determining the pose information of the control device in the display screen coordinate system based on the positioning point coordinates and the angle difference includes:

[0036] Based on the angle difference, the attitude information of the control device is determined in the coordinate system of the display screen;

[0037] Furthermore, the position information of the control device is determined based on the coordinates of the positioning point and the shape of the control device;

[0038] The pose information is generated based on the location information and the attitude information.

[0039] In one embodiment, before the steps of receiving the control device angle from the control device sensor and obtaining the display screen angle from the display screen sensor, the method further includes:

[0040] Obtain the virtual scene to be displayed;

[0041] The virtual scene is rendered using a rendering camera to generate a left-eye rendered image and a right-eye rendered image of the virtual scene.

[0042] The observation position of the human eye is determined, and through parallax analysis of the observation position, combined with the left-eye rendered image and the right-eye rendered image, the image is restored and displayed on the screen.

[0043] In one embodiment, the step of calculating the angle difference between the control device angle and the display screen angle further includes:

[0044] The display screen pose information is updated in real time based on the display screen angle.

[0045] Based on the updated display pose information and the angle of the control device, the angle difference between the control device and the display is calculated.

[0046] In one embodiment, both the control device sensor and the display sensor include a gyroscope and a geomagnetic sensor.

[0047] In one embodiment, the control device angle includes the pitch angle, yaw angle, and roll angle of the control device, and the display screen angle includes the pitch angle, roll angle, and yaw angle of the display screen.

[0048] In addition, to achieve the above objectives, this application also provides a pose determination device for a display screen control device, the device comprising: a memory, a processor, and a computer program stored in the memory and executable on the processor, the computer program being configured to implement the steps of the pose determination method for the display screen control device as described above.

[0049] In addition, to achieve the above objectives, this application also provides a storage medium, which is a computer-readable storage medium, and stores a computer program thereon. When the computer program is executed by a processor, it implements the steps of the pose determination method for the display screen control device as described above. Beneficial effects

[0050] One or more technical solutions proposed in this application have at least the following technical effects:

[0051] This application uses sensors in the control device and the display screen to acquire the angles of the control device and the display screen, respectively, calculates the angle difference, and obtains the coordinates of the control device's positioning point using a positioning sensor. This allows for the determination of the relative pose of the control device and the display screen in space, avoiding interference from other factors in space. Furthermore, determining the control device's pose information based on the relative pose of the control device and the display screen in space avoids the need for recalibration of the display screen after each position change, reducing the complexity of the operation. (See attached figures.)

[0052] The accompanying drawings, which are incorporated in and form part of this specification, illustrate embodiments consistent with this application and, together with the description, serve to explain the principles of this application.

[0053] To more clearly illustrate the technical solutions in the embodiments of this application or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, for those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0054] Figure 1 is a flowchart illustrating the first embodiment of the pose determination method for the display screen control device of this application.

[0055] Figure 2 is a flowchart illustrating the second embodiment of the pose determination method for the display screen control device of this application.

[0056] Figure 3 is a flowchart illustrating the third embodiment of the pose determination method for the display screen control device of this application.

[0057] Figure 4 is a flowchart illustrating the fourth embodiment of the pose determination method for the display screen control device of this application.

[0058] Figure 5 is a schematic diagram of the pose determination device of the display screen control device in the hardware operating environment involved in the embodiment of this application;

[0059] Figure 6 is a flowchart illustrating the fifth embodiment of the pose determination method for the display screen control device of this application.

[0060] The realization of the purpose, functional features and advantages of this application will be further explained in conjunction with the embodiments and with reference to the accompanying drawings. Embodiments of the present invention

[0061] It should be understood that the specific embodiments described herein are merely illustrative of this application and are not intended to limit this application.

[0062] To better understand the above technical solutions, the following will provide a detailed explanation of the technical solutions in conjunction with the accompanying drawings and specific implementation methods.

[0063] The main solution of this application embodiment is: receiving the control device angle collected by the control device sensor and obtaining the display screen angle through the display screen sensor; calculating the angle difference between the control device angle and the display screen angle; identifying the positioning point coordinates of the control device through the positioning sensor, and determining the pose information of the control device based on the angle difference and the positioning point coordinates.

[0064] To prevent the pen's pose from shifting during tracking and positioning, current technology allows VR displays to calibrate the pen's pose by collecting angle information from angle sensors (such as gyroscopes or magnetometers) within the pen. However, to achieve this calibration, the VR display typically needs to determine the pen's pose based on the spatial information of the display's current location when receiving angle information from the angle sensor. Therefore, when the VR display's pose changes, the calibration result obtained after collecting angle information via an infrared camera installed on the VR display deviates from the actual result. This leads to insufficient accuracy in the pen's pose calibration.

[0065] This application uses a control device sensor and a display screen sensor to acquire the angles of the control device and the display screen, respectively, calculates the angle difference, and obtains the coordinates of the control device's positioning point using a positioning sensor. This allows for the determination of the relative pose of the control device and the display screen in space, avoiding interference from other spatial factors in determining the control device's pose. Furthermore, based on the angle information of the control device, its pose information can be determined by collecting data from only a single positioning point, thereby reducing the computational workload for determining the control device's position.

[0066] To better understand the above technical solutions, exemplary embodiments of this application will be described in more detail below with reference to the accompanying drawings. Although exemplary embodiments of this application are shown in the drawings, it should be understood that this application can be implemented in various forms and should not be limited to the embodiments set forth herein. Rather, these embodiments are provided to enable a more thorough understanding of this application and to fully convey the scope of this application to those skilled in the art.

[0067] It should be noted that the executing entity in this embodiment can be a display screen control system, or a computing service device with data processing, network communication, and program execution functions, such as a tablet computer, personal computer, or mobile phone, or an electronic device or a pose determination device for a display screen control device capable of the above functions. This embodiment does not specifically limit the specific implementation. The following uses a display screen control system as an example to describe this embodiment and the following embodiments.

[0068] Based on this, this application provides a method for determining the pose of a display screen control device. Referring to FIG1, FIG1 is a flowchart of the first embodiment of the method for determining the pose of a display screen control device according to this application.

[0069] In this embodiment, the pose determination method of the display screen control device includes steps S10~S30:

[0070] Step S10: Receive the control device angle collected by the control device sensor, and obtain the display screen angle through the display screen sensor;

[0071] In this embodiment, the display screen control system can be deployed on the display screen, acquiring display screen information and performing corresponding control actions on the display screen. Alternatively, the display screen control system can also be deployed as an application on a host device, receiving display screen information transmitted by the display screen via wired or wireless communication, and sending corresponding control commands to the display screen to complete the control actions of the display screen. The display screen can be a VR display screen, an AR display screen, or other types of display screen. The control device can be a control pen, a remote control, or other similar devices.

[0072] In one embodiment, to enable the display screen control device to collect attitude information of the control device itself, and to complete the posture-based control process of the display screen, both the control device and the display screen include angle sensors, namely, control device sensors and display screen sensors. These angle sensors include geomagnetic sensors and gyroscopes, among other sensing devices. The control device can collect its own angle through the control device sensors and send this angle to the display screen control system. Simultaneously, the display screen control system can acquire the display screen angle collected by the display screen sensors.

[0073] In one example, both the display screen and the control device incorporate gyroscopes and geomagnetic sensors in their angle sensors. The gyroscope, composed of a gravity sensor and an accelerometer, determines the device's rotational state by measuring its rotational speed in three dimensions, thus enabling the detection of rotational motion and attitude changes. The geomagnetic sensor, used to determine the device's absolute orientation relative to the Earth's magnetic field, detects the components of the geomagnetic field in the device's three dimensions and is typically composed of three mutually perpendicular magnetoresistive sensors. While the gyroscope cannot provide absolute orientation information, and the geomagnetic sensor lacks motion information such as rotational motion and attitude changes, the combined use of the geomagnetic sensor and gyroscope allows for the detection of rotational motion and attitude changes using the gyroscope, and the determination of the device's absolute orientation using the geomagnetic sensor, resulting in more accurate angle and attitude information.

[0074] In another example, since the display screen is typically in a fixed posture during interaction and there is no change in posture, the angle sensor in the display screen can consist only of a geomagnetic sensor to obtain the absolute orientation of the display screen relative to the geomagnetic direction. However, based on the changes in position and posture information of the control device during user operation, the control device sensors need to include both a gyroscope and a geomagnetic sensor to simultaneously measure the absolute orientation, rotational motion, and posture changes of the control device.

[0075] Step S20: Calculate the angle difference between the control device angle and the display screen angle;

[0076] In this application, the control device angle includes the pitch angle, yaw angle, and roll angle of the control device, while the display screen angle includes the pitch angle, roll angle, and yaw angle of the display screen. The display screen control system can calculate the angle differences between the pitch angle, roll angle, and yaw angle in both the control device angle and the display screen angle to determine the relative attitude between the display screen and the control device. The pitch angle, roll angle, and yaw angle represent the rotation angles of the device around the X, Y, and Z axes of the spatial coordinate system, respectively. The yaw angle needs to be determined by obtaining the device's orientation information relative to the geomagnetic direction using a geomagnetic sensor, while the pitch angle and roll angle can be obtained using sensing devices such as gyroscopes or gravity sensors.

[0077] For example, when the control device is a handheld controller, the display control system can simulate a steering wheel based on the angle of the angle sensor in the handheld controller. The user can change the angle information acquired by the angle sensor by rotating the handheld controller. Upon receiving the controller angle, the display control system can determine the rotation angle of the steering wheel in the virtual scene based on the yaw and roll angles, thus realizing a virtual driving simulation scenario.

[0078] Step S30: Identify the coordinates of the positioning point of the control device using the positioning sensor, and determine the pose information of the control device based on the angle difference and the coordinates of the positioning point.

[0079] It should be noted that pose information consists of position information and attitude information, including the object's six degrees of freedom (6DOF) in space, namely the linear movement of the object in the X, Y, and Z axes, and the rotation of the object around the X, Y, and Z axes, which correspond to the changes in pitch angle, roll angle, and yaw angle, respectively.

[0080] In this embodiment, the display screen and / or control device are further equipped with a positioning sensor, which can determine the position information of the control device in the three-dimensional space where the display screen is located. This position information is determined based on three-dimensional coordinates, including the X, Y, and Z axes. The display screen control system can combine the position and attitude information of the display screen and / or control device to generate corresponding pose information. When the attitude of the control device is determined based on angle differences, the display screen control system only needs to calculate one positioning coordinate of the control device, i.e., the positioning point coordinate, to calculate the corresponding pose information.

[0081] In one embodiment, the display screen control system establishes a display screen coordinate system based on the display screen pose. When acquiring the display screen coordinates, the display screen control system can map the pose of the control device into the display screen coordinate system based on the angle difference between the control device and the display screen, as well as the positioning point coordinates of the control device.

[0082] Specifically, referring to Figure 6, which is a flowchart of the fifth embodiment of the pose determination method for the display screen control device of this application, step S30 includes steps S31 to S33:

[0083] Step S31: Receive the positioning signal from the control device via the positioning sensor;

[0084] Step S32: Determine the coordinates of the positioning point of the control device in the display screen coordinate system based on the positioning signal;

[0085] In the first embodiment, the display screen control system can acquire a positioning image, including a positioning recognition light spot based on the positioning point of the control device, using a light sensor. The display screen control system can further calculate the positioning coordinates of the positioning point in the control device based on the position of the positioning recognition light spot within the positioning image.

[0086] Specifically, the control device can set up a signal transmitter at the positioning point and send a positioning signal through the signal transmitter, forming a positioning recognition light spot in the positioning image acquired by the optical sensor. Alternatively, the control device can also reflect the signal through the positioning point, for example, by reflecting the positioning signal sent by the signal transmitter to the optical sensor through the positioning point, forming a positioning recognition light spot in the positioning image, achieving a similar effect to the signal transmission from the positioning point.

[0087] In one embodiment, to avoid the problem of insufficient positioning accuracy of a single optical sensor, multiple optical sensors are usually used. Based on triangulation or visual positioning algorithms, the actual coordinates of the positioning point are calculated according to information such as image acquisition time, signal strength, and imaging position of the positioning and identification light spot.

[0088] For example, the display screen acquires positioning images of the control device using at least two light sensors, and determines the position of the positioning identification light spot mapped onto the positioning image based on the positioning identification light spot identified in the positioning image. Simultaneously, the display screen control system can obtain the sensor coordinates of the light sensors in the display screen coordinate system. Based on these sensor coordinates and the positional differences of the positioning identification light spot in different positioning images, the display screen control system determines the positioning point coordinates in the display screen coordinate system using triangulation.

[0089] In one embodiment, the display control system may also receive a positioning signal emitted by the control device based on a positioning point, and determine the incident angle and distance of the positioning signal according to the positioning signal. Based on the incident angle and / or distance, and the sensor coordinates of the light sensor in the display coordinate system, the display control system can determine the coordinates of the positioning point.

[0090] In the second embodiment, the display screen control system can also control the relative position of the device and the display screen based on image processing. By converting the acquired image to grayscale to reduce image complexity, performing binarization to convert the image into an image containing only black and white, detecting feature points of the target object, extracting feature points, forming feature vectors, and thus determining the target's coordinates in space.

[0091] In the third embodiment, the display screen control system can also use ultra-wideband (UWB) signals to determine the signal transmission time from when the control device sends a UWB signal to when the display screen receives the UWB signal, thereby determining the distance between the UWB radar and the control device's UWB transmission location, and thus determining the coordinates of the control device's positioning point.

[0092] In one embodiment, based on positioning sensors, the display control system typically determines the relative position of the control device and the display screen. This is because the coordinates of the control device's positioning point in the display screen coordinate system can be directly determined based on this relative position, without considering the influence of environmental factors. The display control system can establish a mapping relationship based on the positions of the positioning point in the display screen coordinate system and its position on the control device, and based on pose information and shape information such as the shape and length of the control device, map the complete position of the control device onto the display screen coordinate system.

[0093] Step S33: Determine the pose information of the control device in the display screen coordinate system based on the coordinates of the positioning point and the angle difference.

[0094] In this embodiment, the display screen control system determines the position information of the control device based on the coordinates of the positioning point and the shape of the control device, and determines the attitude information of the control device in the display screen coordinate system based on the angle difference. Pose information is then generated based on the position information and attitude information.

[0095] This application embodiment uses a control device sensor in the control device and a display screen sensor in the display screen to obtain the angles of the control device and the display screen, respectively, calculate the angle difference, and obtain the coordinates of the positioning point of the control device through a positioning sensor. This allows for the determination of the relative pose of the control device and the display screen in space, avoiding interference from other factors in space. Furthermore, determining the pose information of the control device based on the relative pose of the control device and the display screen in space avoids the need for recalibration of the display screen every time its position is moved, thus reducing the complexity of the operation.

[0096] Based on the same inventive concept, this application also provides a second embodiment. Referring to FIG2, FIG2 is a flowchart illustrating the second embodiment of the pose determination method for the display screen control device of this application.

[0097] In this embodiment, after identifying the coordinates of the positioning point of the control device through the positioning sensor as described in step S30, and determining the pose information of the control device based on the angle difference and the coordinates of the positioning point, the method further includes steps S41 to S44:

[0098] Step S41: Receive the control signal from the control device;

[0099] Step S42: Determine the control command for the control device based on the pose information and the control signal;

[0100] In this embodiment, the display screen control system can determine the control command of the control device based on the position and pose information of the control device and / or other control actions on the control device, and realize the interactive action corresponding to the control command in the virtual scene.

[0101] In one embodiment, the display screen determines the control line of the control device to the virtual scene based on the extension line of the control device, and displays the control line on the display screen. When the user views the display screen, they can see the control line and determine the target in the virtual scene currently being controlled by the control device based on the control line.

[0102] Specifically, the display screen control system determines the extension line of the control device based on the relative position of the control device and the display screen, identifies a target extension line within the virtual scene, and uses this target extension line as the control line of the virtual scene. This target extension line is the line segment of the control device's extension line that passes through the virtual scene. The display screen control system then displays this control line within the virtual scene shown on the display screen.

[0103] Step S43: Determine the interactive action corresponding to the control command in the virtual scene;

[0104] Step S44: Execute the interactive action and display the interactive action and the interaction result of the interactive action in the virtual scene on the display screen.

[0105] In this embodiment, based on user actions such as pressing on the control device, the control device can send corresponding control signals to the display screen control system. The display screen control system can determine the actual control command of the control device based on the control signal and the position information of the control device, and determine the corresponding interactive action. The display screen control system will execute this interactive action in the virtual scene and display the interactive action and the interaction result in the virtual scene as information to be displayed on the display screen.

[0106] For example, the control device may also include buttons. The user moves the device onto an object in the virtual scene using a control line and then moves the object by clicking the buttons on the control device.

[0107] This application embodiment determines the control command of the control device by using the position and posture information of the control device in space and the user's control actions such as clicking buttons on the control device, and completes the interaction with the virtual scene on the display screen.

[0108] Since the system described in Embodiment 2 of this application is a system used to implement the method of Embodiment 1 of this application, those skilled in the art can understand the specific structure and variations of the system based on the method described in Embodiment 1 of this application, and therefore will not be described again here. All systems used in the method of Embodiment 1 of this application fall within the scope of protection of this application.

[0109] Based on the same inventive concept, this application also provides a third embodiment. Referring to FIG3, FIG3 is a flowchart illustrating the third embodiment of the pose determination method for the display screen control device of this application.

[0110] In this embodiment, before receiving the control device angle collected by the control device sensor as described in step S10, and before obtaining the display screen angle through the display screen sensor, steps S11 to S14 are further included:

[0111] Step S11: Obtain the virtual scene to be displayed;

[0112] Step S12: Render the virtual scene using a rendering camera to generate a left-eye rendered image and a right-eye rendered image of the virtual scene;

[0113] Step S13: Determine the human eye observation position, and through parallax analysis of the human eye observation position, combine the left eye rendered image and the right eye rendered image to perform image restoration display on the display screen.

[0114] In this embodiment, the display screen supports displaying virtual scenes in a glasses-free 3D manner. Based on the characteristic that the images seen by the left and right eyes are not entirely identical from different perspectives, the display screen control system can render separate left-eye and right-eye rendered images according to the virtual scene and the corresponding display angle. Specifically, the display screen control system uses a rendering camera to simulate the left-eye and right-eye lines of view respectively, generating a left-eye rendered image from the left-eye perspective and a right-eye rendered image from the right-eye perspective.

[0115] In one embodiment, the display screen determines the user's eye observation position by capturing the user's facial image through sensors or by using light-sensing recognition from the glasses. This eye observation position is the spatial coordinate of the eyeball within the display screen's coordinate system. The display screen further performs parallax analysis on the eye observation position to determine how to combine the left-eye and right-eye rendered images to reconstruct the rendered image. The combined rendered image is then displayed on the screen, allowing the user to see different images with each eye, creating a stereoscopic effect and forming a naked-eye 3D effect.

[0116] In this embodiment, the display screen renders a left-eye rendered image and a right-eye rendered image respectively. By analyzing the parallax of the two eyes, the rendered image of the virtual scene is analyzed, and the left-eye rendered image and the right-eye rendered image are combined for restoration and display, thereby forming a naked-eye 3D effect.

[0117] Based on the same inventive concept, this application also provides a fourth embodiment. Referring to FIG4, FIG4 is a flowchart illustrating the fourth embodiment of the pose determination method for the display screen control device of this application.

[0118] In this embodiment, the pose determination method of the display screen control device further includes steps S51-S52:

[0119] Step S51: Update the display pose information of the display screen in real time according to the angle of the display screen;

[0120] Step S52: Based on the updated display pose information and the angle of the control device, calculate the angle difference between the control device and the display.

[0121] In this embodiment, in order to reduce the computational load of the display screen control system based on the pose, the display screen control system can correct the display screen pose and / or the display screen coordinate system based on the display screen pose at preset intervals.

[0122] In one embodiment, the display screen control system can track and locate the control device based on changes in its pose in space. Specifically, the display screen control system can determine the pose of the control device solely based on its angles and the coordinates of the positioning point, provided the display screen pose or the X, Y, and Z axis angles of the display screen coordinate system are known. This reduces the continuous computational load on the control device's pose during the positioning and tracking process.

[0123] Specifically, the display screen control system uses display screen sensors to detect changes in the display screen angle in real time. When the display screen angle changes, it triggers an update process for the display screen's pose information. It acquires the display screen angle, calculates the current pose information based on the angle, and replaces the currently stored pose information, thus updating the pose information and / or the X, Y, and Z axis angles of the display screen coordinate system. When the display screen angle remains unchanged, it calculates the angle difference and the pose of the control device using the original display screen pose and / or the existing angle.

[0124] In one embodiment, the display control system may also trigger the display pose information update process when the angle information changes, based on angle sensing devices such as gyroscopes, and update the display pose information or the angles of the X, Y, and Z axes in the display coordinate system based on the changed angle information.

[0125] This application provides a pose determination device for a display screen control device, the device comprising: at least one processor; and a memory communicatively connected to the at least one processor; wherein the memory stores instructions executable by the at least one processor, the instructions being executed by the at least one processor to enable the at least one processor to perform the pose determination method for the display screen control device described in Embodiment 1 above.

[0126] Referring now to Figure 5, a schematic diagram of a pose determination device suitable for implementing the display screen control device of this application is shown. The pose determination device for the display screen control device in this application embodiment may include, but is not limited to, mobile terminals such as mobile phones, laptops, digital broadcast receivers, PDAs (Personal Digital Assistants), PADs (Portable Application Description), PMPs (Portable Media Players), in-vehicle terminals (e.g., in-vehicle navigation terminals), and fixed terminals such as digital TVs and desktop computers. The pose determination device for the display screen control device shown in Figure 5 is merely an example and should not impose any limitations on the functionality and scope of use of the embodiments of this application.

[0127] As shown in Figure 5, the pose determination device of the display screen control device may include a processing unit 1001 (e.g., a core processor, a graphics processor, etc.), which can perform various appropriate actions and processes according to a program stored in read-only memory (ROM) 1002 or a program loaded from storage device 1003 into random access memory (RAM) 1004. The RAM 1004 also stores various programs and data required for the operation of the pose determination device of the display screen control device. The processing unit 1001, the read-only memory (ROM) 1002, and the random access memory (RAM) 1004 are interconnected via a bus 1005. An input / output (I / O) interface 1006 is also connected to the bus. Typically, the following systems can be connected to I / O interface 1006: input devices 1007 including, for example, touchscreens, touchpads, keyboards, mice, image sensors, microphones, accelerometers, gyroscopes, etc.; output devices 1008 including, for example, liquid crystal displays (LCDs), speakers, vibrators, etc.; storage devices 1003 including, for example, magnetic tapes, hard disks, etc.; and communication devices 1009. Communication device 1009 allows the pose determination device of the display control device to wirelessly or wiredly communicate with other devices to exchange data. Although the figure shows a pose determination device of a display control device with various systems, it should be understood that it is not required to implement or possess all the systems shown. More or fewer systems can be implemented alternatively.

[0128] Specifically, according to the embodiments disclosed in this application, the processes described above with reference to the flowcharts can be implemented as computer software programs. For example, embodiments disclosed in this application include a computer program product comprising a computer program carried on a computer-readable medium, the computer program containing program code for performing the methods shown in the flowcharts. In such embodiments, the computer program can be downloaded and installed from a network via a communication device, or installed from storage device 1003, or installed from read-only memory (ROM) 1002. When the computer program is executed by processing device 1001, it performs the functions defined in the methods of the embodiments disclosed in this application.

[0129] The pose determination device for a display screen control device provided in this application, employing the pose determination method for a display screen control device in the above embodiments, can solve the technical problem of insufficient accuracy in the pose calibration results of the VR display screen through the angle sensor in the control pen. Compared with the prior art, the beneficial effects of the pose determination device for a display screen control device provided in this application are the same as those of the pose determination method for a display screen control device provided in the above embodiments, and other technical features in the pose determination device for a display screen control device are the same as those disclosed in the method of the previous embodiment, and will not be repeated here.

[0130] It should be understood that the various parts disclosed in this application can be implemented using hardware, software, firmware, or a combination thereof. In the description of the above embodiments, specific features, structures, materials, or characteristics can be combined in any suitable manner in one or more embodiments or examples.

[0131] The above description is merely a specific embodiment of this application, but the scope of protection of this application is not limited thereto. Any variations or substitutions that can be easily conceived by those skilled in the art within the scope of the technology disclosed in this application should be included within the scope of protection of this application. Therefore, the scope of protection of this application should be determined by the scope of the claims.

[0132] This application provides a computer-readable storage medium having computer-readable program instructions (i.e., a computer program) stored thereon, the computer-readable program instructions being used to execute the pose determination method of the display screen control device in the above embodiments.

[0133] The computer-readable storage medium provided in this application may be, for example, a USB flash drive, but is not limited to, electrical, magnetic, optical, electromagnetic, infrared, or semiconductor systems, devices, or any combination thereof. More specific examples of computer-readable storage media may include, but are not limited to: electrical connections having one or more wires, portable computer disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fiber, portable compact disk read-only memory (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination thereof. In this embodiment, the computer-readable storage medium may be any tangible medium containing or storing a program that can be used by or in conjunction with an instruction execution system, system, or device. The program code contained on the computer-readable storage medium may be transmitted using any suitable medium, including but not limited to: wires, optical cables, RF (Radio Frequency), etc., or any suitable combination thereof.

[0134] The aforementioned computer-readable storage medium may be included in the pose determination device of the display control device; or it may exist independently and not assembled into the pose determination device of the display control device.

[0135] The aforementioned computer-readable storage medium carries one or more programs that, when executed by the pose determination device of the display screen control device, cause the pose determination device of the display screen control device to: receive the control device angle collected by the control device sensor and obtain the display screen angle through the display screen sensor; calculate the angle difference between the control device angle and the display screen angle; identify the positioning point coordinates of the control device through the positioning sensor, and determine the pose information of the control device based on the angle difference and the positioning point coordinates.

[0136] Computer program code for performing the operations of this application can be written in one or more programming languages ​​or a combination thereof, including object-oriented programming languages ​​such as Java, Smalltalk, and C++, and conventional procedural programming languages ​​such as the "C" language or similar programming languages. The program code can be executed entirely on the user's computer, partially on the user's computer, as a standalone software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In cases involving remote computers, the remote computer can be connected to the user's computer via any type of network—including a Local Area Network (LAN) or a Wide Area Network (WAN)—or can be connected to an external computer (e.g., via the Internet using an Internet service provider).

[0137] The flowcharts and block diagrams in the accompanying drawings illustrate the architecture, functionality, and operation of possible implementations of systems, methods, and computer program products according to various embodiments of this application. In this regard, each block in a flowchart or block diagram may represent a module, segment, or portion of code containing one or more executable instructions for implementing a specified logical function. It should also be noted that in some alternative implementations, the functions indicated in the blocks may occur in a different order than those indicated in the drawings. For example, two consecutively indicated blocks may actually be executed substantially in parallel, and they may sometimes be executed in reverse order, depending on the functions involved. It should also be noted that each block in the block diagrams and / or flowcharts, and combinations of blocks in the block diagrams and / or flowcharts, can be implemented using a dedicated hardware-based system that performs the specified function or operation, or using a combination of dedicated hardware and computer instructions.

[0138] The modules described in the embodiments of this application can be implemented in software or hardware. The names of the modules do not necessarily limit the functionality of the unit itself.

[0139] The readable storage medium provided in this application is a computer-readable storage medium that stores computer-readable program instructions (i.e., a computer program) for executing the pose determination method of the above-described display control device. This solves the technical problem of insufficient accuracy in the pose calibration results of the VR display pen via the angle sensor in the pen. Compared with the prior art, the beneficial effects of the computer-readable storage medium provided in this application are the same as those of the pose determination method of the display control device provided in the above embodiments, and will not be repeated here.

[0140] The above description is only a part of the embodiments of this application and does not limit the patent scope of this application. All equivalent structural transformations made under the technical concept of this application and using the contents of the specification and drawings of this application, or direct / indirect applications in other related technical fields, are included in the patent protection scope of this application.

Claims

1. A method for determining the pose of a display screen control device, wherein, The method for determining the pose of the display screen control device includes the following steps: The angle of the control device is acquired by the sensor of the control device, and the angle of the display screen is acquired by the sensor of the display screen; Calculate the angle difference between the control device angle and the display screen angle; The positioning sensors identify the coordinates of the positioning points of the control device, and the position and pose information of the control device is determined based on the angle difference and the coordinates of the positioning points.

2. The method as described in claim 1, wherein, After the step of identifying the coordinates of the positioning point of the control device through the positioning sensor and determining the pose information of the control device based on the angle difference and the coordinates of the positioning point, the method further includes: Receive control signals from the control device; Based on the pose information and the control signal, the control command of the control device is determined; Determine the interactive actions corresponding to the control commands in the virtual scene; The interactive action is executed, and the interactive action and its result in the virtual scene are displayed on the screen.

3. The method as described in claim 2, wherein, After the step of identifying the coordinates of the positioning point of the control device through the positioning sensor and determining the pose information of the control device based on the angle difference and the coordinates of the positioning point, the method further includes: Based on the relative pose of the control device and the display screen, determine the extension line of the control device; Determine the target extension line of the extension line in the virtual scene, and use the target extension line as the control line of the virtual scene; The control line is displayed in the virtual scene shown on the display screen.

4. The method of claim 1, wherein, The step of identifying the coordinates of the positioning point of the control device through the positioning sensor, and determining the pose information of the control device based on the angle difference and the coordinates of the positioning point, includes: The positioning signal of the control device is received through the positioning sensor; The coordinates of the control device's positioning point in the display screen coordinate system are determined based on the positioning signal. Based on the coordinates of the positioning point and the angle difference, the position and pose information of the control device in the coordinate system of the display screen is determined.

5. The method of claim 4, wherein, The step of determining the coordinates of the control device's positioning point in the display screen coordinate system based on the positioning signal includes: The positioning image of the control device is acquired using at least two optical sensors; Identify the positioning recognition light spot of the control device based on the positioning point in the positioning image; The coordinates of the positioning point are determined by triangulation based on the position of the positioning spot mapped onto the positioning image and the sensor coordinates of the light sensor in the display screen coordinate system.

6. The method of claim 4, wherein, The step of determining the coordinates of the control device's positioning point in the display screen coordinate system based on the positioning signal includes: Based on image processing, the relative position of the control device and the display screen is determined, and the image acquired by the display screen control system is converted to grayscale to obtain a grayscale image. The image after grayscale processing is binarized to obtain a binarized image; Feature point detection is performed on the target object in the binarized image, feature points are extracted, and feature vectors are formed to determine the coordinates of the target in space.

7. The method of claim 4, wherein, The step of determining the coordinates of the control device's positioning point in the display screen coordinate system based on the positioning signal includes: The display screen control system uses ultra-wideband signals to determine the signal transmission time from when the control device sends the ultra-wideband signal to when the display screen receives the ultra-wideband signal, thereby determining the distance between the ultra-wideband radar and the ultra-wideband transmission position of the control device, and thus determining the positioning point coordinates of the control device.

8. The method of claim 4, wherein, The step of determining the pose information of the control device in the display screen coordinate system based on the positioning point coordinates and the angle difference includes: Based on the angle difference, the attitude information of the control device is determined in the coordinate system of the display screen; Furthermore, the position information of the control device is determined based on the coordinates of the positioning point and the shape of the control device; The pose information is generated based on the location information and the attitude information.

9. The method of claim 1, wherein, Before the steps of receiving the control device angle from the control device sensor and obtaining the display screen angle from the display screen sensor, the method further includes: Obtain the virtual scene to be displayed; The virtual scene is rendered using a rendering camera to generate a left-eye rendered image and a right-eye rendered image of the virtual scene. The observation position of the human eye is determined, and through parallax analysis of the observation position, combined with the left-eye rendered image and the right-eye rendered image, the image is restored and displayed on the screen.

10. The method of claim 1, wherein, The step of calculating the angle difference between the control device angle and the display screen angle further includes: The display screen pose information is updated in real time based on the display screen angle. Based on the updated display pose information and the angle of the control device, the angle difference between the control device and the display is calculated.

11. The method of claim 1, wherein, Both the control device sensor and the display screen sensor include a gyroscope and a geomagnetic sensor.

12. The method of claim 1, wherein, The control device angles include the pitch angle, yaw angle, and roll angle of the control device, and the display screen angles include the pitch angle, roll angle, and yaw angle of the display screen.

13. A pose determination device for a display screen control device, wherein, The device includes: a memory, a processor, and a computer program stored in the memory and executable on the processor, the computer program being configured to implement the steps of the pose determination method for the display control device as described in any one of claims 1 to 12.

14. A storage medium, wherein, The storage medium is a computer-readable storage medium, and a computer program is stored on the storage medium. When the computer program is executed by a processor, it implements the steps of the pose determination method of the display screen control device as described in any one of claims 1 to 12.

Images