Method, device and storage medium for determining the posture of a display screen control device

Abstract

The present application discloses a method, device and storage medium for determining the posture of a display screen control device, which belongs to the field of spatial positioning technology. The method includes: receiving the control device angle collected by the control device sensor, and obtaining the display screen angle through the display screen sensor, calculating the angle difference between the control device angle and the display screen angle, identifying the positioning point coordinates of the control device through the positioning sensor, and determining the posture information of the control device based on the angle difference and the positioning point coordinates. The present application determines the posture information of the control device by calculating the angle difference between the control device angle of the control device and the display screen angle of the display screen, and obtaining the positioning point coordinates of the control device through the positioning sensor, obtaining the relative posture of the control device and the display screen in space.

Description

Technical Field

[0001] The present application relates to the field of spatial positioning technology, and in particular to a method, device, and storage medium for determining the posture of a display screen control device. Background Art

[0002] To enable interaction between users and virtual scenes, VR displays usually use infrared cameras to collect the spatial coordinates of multiple positioning points in the control pen to form the position of the control pen in space.

[0003] In related technologies, in order to prevent the posture of the control pen from being offset during the tracking and positioning process, the VR display screen can collect the angle information of the control pen through angle sensors in the control pen, such as gyroscopes and geomagnetic sensors, and calibrate the posture of the control pen.

[0004] However, to calibrate the pen's position using the angle sensor, the VR display typically needs to determine the pen's position based on the display's current spatial information when receiving angle information from the pen's angle sensor. Consequently, if the VR display's position changes, the pen's position calibration result, collected via the infrared camera installed on the VR display, deviates from the actual result. This results in inaccurate pen calibration results.

[0005] The above content is only used to assist in understanding the technical solution of this application and does not constitute an admission that the above content is prior art. Summary of the Invention

[0006] The main purpose of this application is to provide a method, device and storage medium for determining the posture of a display screen control device, aiming to solve the technical problem of insufficient accuracy of the posture calibration results of the control pen by the angle sensor in the control pen of the VR display screen.

[0007] To achieve the above objectives, the present application provides a method for determining the posture of a display screen control device, the method comprising the following steps:

[0008] Receive the control device angle collected by the control device sensor, and obtain the display screen angle through the display screen sensor;

[0009] Calculating the angle difference between the control device angle and the display screen angle;

[0010] The positioning point coordinates of the control device are identified by a positioning sensor, and the posture information of the control device is determined according to the angle difference and the positioning point coordinates.

[0011] In one embodiment, after the step of identifying the positioning point coordinates of the control device by a positioning sensor and determining the posture information of the control device based on the angle difference and the positioning point coordinates, the method further includes:

[0012] receiving a control signal from the control device;

[0013] Determining a control instruction of the control device according to the posture information and the control signal;

[0014] Determining an interactive action corresponding to the control instruction in the virtual scene;

[0015] The interactive action is executed, and the interactive action and the interactive result of the interactive action in the virtual scene are displayed on a display screen.

[0016] In one embodiment, after the step of identifying the positioning point coordinates of the control device by a positioning sensor and determining the posture information of the control device based on the angle difference and the positioning point coordinates, the method further includes:

[0017] determining an extension line of the control device according to the relative posture of the control device and the display screen;

[0018] determining a target extension line of the extension line in the virtual scene, and using the target extension line as a control line of the virtual scene;

[0019] The control line is displayed in the virtual scene displayed on the display screen.

[0020] In one embodiment, the step of identifying the coordinates of the positioning point of the control device by a positioning sensor and determining the posture information of the control device according to the angle difference and the positioning point coordinates includes:

[0021] receiving a positioning signal of the control device through a positioning sensor;

[0022] Determine the coordinates of the positioning point of the control device in the display screen coordinate system based on the positioning signal;

[0023] The position information of the control device in the display screen coordinate system is determined according to the positioning point coordinates and the angle difference.

[0024] In one embodiment, the step of determining the coordinates of the positioning point of the control device in the display screen coordinate system based on the positioning signal includes:

[0025] collecting a positioning image of the control device by at least two optical sensors;

[0026] Identify a positioning recognition light spot of the control device based on the positioning point in the positioning image;

[0027] The coordinates of the positioning point are determined by triangulation according to the position of the positioning recognition light spot mapped to the positioning image and the sensor coordinates of the light sensor in the display screen coordinate system.

[0028] In one embodiment, the step of determining the position information of the control device in the display screen coordinate system based on the positioning point coordinates and the angle difference includes:

[0029] Determining, in the display screen coordinate system, posture information of the control device based on the angle difference;

[0030] and, determining the position information of the control device according to the coordinates of the positioning point and the shape of the control device;

[0031] The posture information is generated based on the position information and the posture information.

[0032] In one embodiment, before the steps of receiving the control device angle acquired by the control device sensor and acquiring the display screen angle through the display screen sensor, the method further includes:

[0033] Obtaining a virtual scene to be displayed;

[0034] Performing image rendering on the virtual scene by using a rendering camera to generate a left-eye rendered image and a right-eye rendered image of the virtual scene;

[0035] The observation position of the human eye is determined, and through parallax analysis of the observation position of the human eye, the left-eye rendered image and the right-eye rendered image are combined to perform image restoration and display on a display screen.

[0036] In one embodiment, the step of calculating the angle difference between the control device angle and the display screen angle further comprises:

[0037] According to the display screen angle, the display screen posture information of the display screen is updated in real time;

[0038] The angle difference between the control device and the display screen is calculated based on the updated display screen posture information and the control device angle.

[0039] In addition, to achieve the above-mentioned purpose, the present application also provides a posture determination device for a display screen control device, the device comprising: a memory, a processor, and a computer program stored on the memory and executable on the processor, the computer program being configured to implement the steps of the posture determination method for the display screen control device as described above.

[0040] In addition, to achieve the above-mentioned purpose, the present application also provides a storage medium, which is a computer-readable storage medium, and a computer program is stored on the storage medium. When the computer program is executed by the processor, the steps of the posture determination method of the display screen control device as described above are implemented.

[0041] One or more technical solutions proposed in this application have at least the following technical effects:

[0042] This application uses a control device sensor in the control device and a display screen sensor in the display screen to obtain the control device angle and the display screen angle respectively, calculate the angle difference, and obtain the positioning point coordinates of the control device through a positioning sensor, thereby obtaining the relative position of the control device and the display screen in space, while avoiding interference from other factors in the space in determining the position of the control device. At the same time, determining the position information of the control device based on the relative position of the control device and the display screen in space can avoid the need to recalibrate the display screen every time the display screen moves, avoiding overly cumbersome operations. BRIEF DESCRIPTION OF THE DRAWINGS

[0043] The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments consistent with the present application and, together with the description, serve to explain the principles of the present application.

[0044] In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the following briefly introduces the drawings required for use in the embodiments or the description of the prior art. Obviously, for ordinary technicians in this field, other drawings can be obtained based on these drawings without any creative work.

[0045] Figure 1 This is a flow chart of the first embodiment of the method for determining the posture of a display screen control device of the present application;

[0046] Figure 2 This is a flow chart of a second embodiment of a method for determining the posture of a display screen control device of the present application;

[0047] Figure 3 This is a flow chart of a third embodiment of a method for determining a posture of a display screen control device of the present application;

[0048] Figure 4 This is a flow chart of a fourth embodiment of a method for determining a posture of a display screen control device of the present application;

[0049] Figure 5 It is a structural diagram of a posture determination device of a display screen control device in a hardware operating environment involved in an embodiment of the present application.

[0050] The realization of the objectives, functional features and advantages of this application will be further explained in conjunction with embodiments and with reference to the accompanying drawings. DETAILED DESCRIPTION

[0051] It should be understood that the specific embodiments described herein are only used to explain the present application and are not intended to limit the present application.

[0052] In order to better understand the above technical solution, the above technical solution will be described in detail below with reference to the accompanying drawings and specific implementation methods.

[0053] The main solution of the embodiment of the present application is: receiving the control device angle collected by the control device sensor, and obtaining the display screen angle through the display screen sensor; calculating the angle difference between the control device angle and the display screen angle; identifying the positioning point coordinates of the control device through the positioning sensor, and determining the posture information of the control device based on the angle difference and the positioning point coordinates.

[0054] In order to prevent the posture of the control pen from shifting during the tracking and positioning process, the existing technology allows the VR display to collect the angle information of the control pen through angle sensors such as gyroscopes and geomagnetic sensors in the control pen, and calibrate the posture of the control pen. However, in order to calibrate the posture of the control pen by the angle sensor, when the VR display receives the angle information of the angle sensor in the control pen, it is usually necessary to determine the posture of the control pen based on the spatial information of the space where the display is currently located. Therefore, in the case of a posture change of the VR display, after the VR display collects the angle information through the infrared camera set in the VR display, there is a deviation between the calibration result of the posture of the control pen and the actual result. This leads to insufficient accuracy of the posture calibration result of the control pen.

[0055] This application uses a control device sensor in the control device and a display screen sensor in the display screen to respectively obtain the control device angle and the display screen angle, calculate the angle difference, and obtain the coordinates of the control device's positioning point through a positioning sensor, thereby obtaining the relative position of the control device and the display screen in space, while avoiding interference from other factors in the space in determining the control device's position. At the same time, based on the angle information of the control device, the position information of the control device can be determined by only collecting a single positioning point of the control device, thereby reducing the amount of calculation of the control device's position information.

[0056] To better understand the above technical solutions, exemplary embodiments of the present application will be described in more detail below with reference to the accompanying drawings. Although exemplary embodiments of the present application are shown in the accompanying drawings, it should be understood that the present application can be implemented in various forms and should not be limited by the embodiments described herein. Rather, these embodiments are provided to enable a more thorough understanding of the present application and to fully convey the scope of the present application to those skilled in the art.

[0057] It should be noted that the execution subject of this embodiment can be a display screen control system, or a computing service device with data processing, network communication, and program execution functions, such as a tablet computer, personal computer, mobile phone, etc., or an electronic device capable of performing the above functions, a display screen control device position determination device, etc., and this embodiment does not specifically limit this. The following uses a display screen control system as an example to illustrate this embodiment and the following embodiments.

[0058] Based on this, the embodiment of the present application provides a method for determining the posture of a display screen control device, referring to Figure 1 , Figure 1 This is a flow chart of the first embodiment of the method for determining the posture of a display screen control device of the present application.

[0059] In this embodiment, the method for determining the posture of the display screen control device includes steps S10 to S30:

[0060] Step S10: receiving the control device angle acquired by the control device sensor, and acquiring the display screen angle through the display screen sensor;

[0061] In this embodiment, the display control system can be deployed in the display screen, can obtain display screen information, and perform corresponding control actions on the display screen. Alternatively, the display control system can also be deployed as an application in the host device, which receives display screen information transmitted by the display screen through wired or wireless communication, and sends corresponding control instructions to the display screen to complete the control actions of the display screen. Among them, the display screen can be selected as a VR display screen, an AR display screen, or other display screens. The control device can be selected as a control pen, a remote control, or other devices.

[0062] Furthermore, to enable the display screen control device to collect the control device's posture information and complete the control device's posture-based control of the display screen, both the control device and the display screen include angle sensors, namely, a control device sensor and a display screen sensor. The angle sensor includes a geomagnetic sensor and other sensing devices, such as a gyroscope. The control device can use the control device sensor to collect the control device's angle and transmit this control device angle to the display screen control system. Simultaneously, the display screen control system can obtain the display screen angle collected by the display screen sensor.

[0063] In one example, the angle sensors in the display screen and the control device both include a gyroscope and a geomagnetic sensor. The gyroscope consists of a gravity sensor and an acceleration sensor. It determines the device's rotational state by measuring the device's rotational speed in three dimensions, thereby enabling the device's rotational motion and attitude changes. The geomagnetic sensor is used to determine the device's absolute orientation relative to the geomagnetic field and can detect the components of the geomagnetic field in the device's three dimensions. It is typically composed of three mutually perpendicular magnetoresistive sensors. The gyroscope cannot provide the device's absolute orientation information, while the geomagnetic sensor lacks motion information such as the device's rotational motion and attitude changes. Therefore, based on the combined use of the geomagnetic sensor and the gyroscope, the device's rotational motion and attitude changes can be detected by the gyroscope, and the device's absolute orientation can be determined by the geomagnetic sensor, obtaining more accurate angle information and attitude information.

[0064] In another example, since the display screen typically maintains a fixed posture during interaction and does not change posture, the display screen's angle sensor can consist solely of a geomagnetic sensor to obtain the display screen's absolute orientation relative to the geomagnetic direction. However, given that the control device's position and posture information changes during user operation, the control device's sensor needs to include both a gyroscope and a geomagnetic sensor to simultaneously measure the control device's absolute orientation, rotational motion, and posture changes.

[0065] Step S20: Calculating the angle difference between the control device angle and the display screen angle;

[0066] In this application, the control device angle includes the pitch angle, yaw angle and roll angle of the control device, and the display screen angle includes the pitch angle, roll angle and yaw angle of the display screen. Among them, the display screen control system can calculate the angle difference between the pitch angle, roll angle and yaw angle in the control device angle and the display screen angle respectively, so as to determine the relative posture between the display screen and the control device. The pitch angle, roll angle and yaw angle respectively represent the rotation angle of the device around the three axes X, Y and Z of the spatial coordinate system. Among them, the yaw angle needs to be determined by the direction information of the device relative to the geomagnetic direction obtained by the geomagnetic sensor, while the pitch angle and roll angle can be obtained by sensing devices such as gyroscopes or gravity sensors.

[0067] For example, when the control device is a joystick, the display control system can simulate a steering wheel based on the angle of the angle sensor in the joystick. The user can change the angle information obtained by the angle sensor by rotating the joystick. Upon receiving the controller angle, the display control system can determine the rotation angle of the steering wheel in the virtual scene based on the yaw and roll angles, thus achieving a virtual driving simulation.

[0068] Step S30: identifying the coordinates of the positioning point of the control device through a positioning sensor, and determining the posture information of the control device according to the angle difference and the coordinates of the positioning point.

[0069] It should be noted that pose information consists of position information and attitude information, including the six degrees of freedom (6DOF) of the object in space, that is, the linear movement of the object in the three axes of X, Y, and Z, and the rotation of the object around the X, Y, and Z axes, which correspond to the changes in pitch angle, roll angle, and yaw angle respectively.

[0070] In this embodiment, a positioning sensor is further provided in the display screen and / or the control device, which can determine the position information of the control device in the three-dimensional space where the display screen is located. The position information is determined based on the three-dimensional coordinates of the space, including the X, Y, and Z axes. The display screen control system can combine the position information and posture information of the display screen and / or the control device to generate corresponding posture information. In the case of determining the posture of the control device based on the angle difference, the display screen control system only needs to calculate one positioning coordinate of the control device, that is, the positioning point coordinate, to calculate the corresponding posture information.

[0071] In one embodiment, the display control system is provided with a display coordinate system based on the display posture. When obtaining the display coordinates, the display control system can map the control device's posture in the display coordinate system based on the angular difference between the control device and the display and the coordinates of the control device's positioning point.

[0072] Specifically, step S30 includes steps S31 to S33:

[0073] Step S31: receiving a positioning signal of the control device through a positioning sensor;

[0074] Step S32: determining the coordinates of the positioning point of the control device in the display screen coordinate system based on the positioning signal;

[0075] In the first optional real-time method, the display control system can use a light sensor to capture a positioning image containing a positioning identification light spot of the control device based on the positioning point. The display control system can further calculate the positioning point coordinates of the positioning point in the control device based on the position of the positioning identification light spot in the positioning image.

[0076] Specifically, the control device can place a signal transmitter at the location of the positioning point and send a positioning signal through the signal transmitter, forming a positioning identification light spot in the positioning image collected by the light sensor. Alternatively, the control device can also use the positioning point to reflect the signal, for example, by reflecting the positioning signal sent by the signal transmitter to the light sensor through the positioning point, forming a positioning identification light spot in the positioning image, achieving an effect similar to the positioning point transmitting the signal.

[0077] Furthermore, to avoid the problem of insufficient positioning accuracy of a single light sensor, multiple light sensors are usually used. Based on triangulation positioning or visual positioning algorithm, the actual coordinates of the positioning point are calculated according to information such as image acquisition time, signal strength, and imaging position of the positioning recognition spot.

[0078] Exemplarily, the display screen uses at least two optical sensors to capture a positioning image of the control device and, based on the positioning identification light spot identified in the positioning image, determines the position of the positioning identification light spot mapped into the positioning image. Simultaneously, the display screen control system can obtain the sensor coordinates of the optical sensors in the display screen coordinate system. Based on these sensor coordinates and the positional differences between the positioning identification light spots in different positioning images, the display screen control system determines the coordinates of the positioning point in the display screen coordinate system through triangulation.

[0079] Optionally, the display control system may also receive a positioning signal transmitted by the control device based on the positioning point and determine the incident angle and distance of the positioning signal based on the positioning signal. The display control system may determine the coordinates of the positioning point based on the incident angle and / or distance and the sensor coordinates of the light sensor in the display coordinate system.

[0080] In a second alternative embodiment, the display screen control system can also control the relative position of the device and the display screen based on image processing. This is accomplished by gray-scaling the captured image to reduce image complexity, performing binarization to convert the image into a black and white image, and then performing feature point detection on the target object. Feature points are extracted to form feature vectors, thereby determining the target's coordinates in space.

[0081] In a third optional embodiment, the display control system can also use an ultra-wideband (UWB) signal to determine the signal transmission time from the control device sending the UWB signal to the display screen receiving the UWB signal, so as to determine the distance between the UWB radar and the UWB sending position of the control device, thereby determining the positioning point coordinates of the control device.

[0082] Furthermore, based on the positioning sensor, the display control system typically determines the relative position of the control device and the display screen. This relative position allows the coordinates of the control device's location point in the display screen coordinate system to be directly determined, without considering the influence of environmental factors. The display control system can establish a mapping relationship based on the position of the location point in the display screen coordinate system and the position on the control device. Based on the posture information and shape information such as the shape and length of the control device, the display control system can map the complete control device position to the display screen coordinate system.

[0083] Step S33: determining the position information of the control device in the display screen coordinate system according to the positioning point coordinates and the angle difference.

[0084] In this embodiment, the display screen control system determines the position information of the control device based on the coordinates of the positioning point and the shape of the control device, and determines the posture information of the control device in the display screen coordinate system based on the angle difference. Based on the position information and posture information, the posture information is generated.

[0085] In the embodiment of the present application, the control device sensor in the control device and the display screen sensor in the display screen respectively obtain the control device angle and the display screen angle, calculate the angle difference, and obtain the positioning point coordinates of the control device through the positioning sensor, thereby obtaining the relative position of the control device and the display screen in space, while avoiding interference from other factors in the space in determining the position of the control device. At the same time, by determining the position information of the control device based on the relative position of the control device and the display screen in space, it is possible to avoid the need to recalibrate the display screen every time the display screen moves, avoiding overly cumbersome operations.

[0086] Based on the same inventive concept, this application also provides a second embodiment, referring to Figure 2 , Figure 2 This is a flow chart of the second embodiment of the method for determining the posture of a display screen control device of the present application.

[0087] In this embodiment, after identifying the positioning point coordinates of the control device by the positioning sensor as described in step S30, and determining the posture information of the control device according to the angle difference and the positioning point coordinates, steps S41 to S44 are also included:

[0088] Step S41: receiving a control signal from the control device;

[0089] Step S42: determining a control instruction of the control device according to the posture information and the control signal;

[0090] In this embodiment, the display screen control system can determine the control instruction of the control device based on the posture information of the control device and / or other control actions on the control device, and implement the interactive action corresponding to the control instruction in the virtual scene.

[0091] Optionally, the display screen determines the control line of the control device on the virtual scene based on the extension line of the control device and displays the control line on the display screen. When the user views the display screen, the control line can be seen and the target in the virtual scene currently being controlled by the control device can be determined based on the control line.

[0092] Specifically, the display screen control system determines an extension line of the control device based on the relative position of the control device and the display screen, determines a target extension line of the extension line in the virtual scene, and uses the target extension line as the control line of the virtual scene. The target extension line is a line segment of the extension line of the control device that passes through the virtual scene. The display screen control system displays the control line in the virtual scene displayed on the display screen.

[0093] Step S43: determining the interactive action corresponding to the control instruction in the virtual scene;

[0094] Step S44: executing the interactive action, and displaying the interactive action and the interactive result of the interactive action in the virtual scene on a display screen.

[0095] In this embodiment, based on a user's control action on the control device, such as a press, the control device can send a corresponding control signal to the display control system. Based on the control signal and the position information of the control device, the display control system can determine the actual control instruction of the control device and the interactive action corresponding to the control instruction. The display control system executes the interactive action in the virtual scene and displays the interactive action and the interactive result in the virtual scene as the information to be displayed on the display screen.

[0096] For example, the control device may further include buttons. The user moves to an object in the virtual scene according to the control line and moves the object in the virtual scene by clicking the buttons on the control device.

[0097] The embodiment of the present application determines the control instructions of the control device through the posture information of the control device in space and the control actions of the user such as clicking buttons on the control device, thereby completing the interaction with the virtual scene on the display screen.

[0098] Since the system described in Example 2 of this application is the system used to implement the method of Example 1 of this application, those skilled in the art will be able to understand the specific structure and variations of the system based on the method described in Example 1 of this application, and therefore, no further description is given here. All systems used in the method of Example 1 of this application fall within the scope of protection to be provided by this application.

[0099] Based on the same inventive concept, this application also provides a third embodiment, referring to Figure 3 , Figure 3 This is a flow chart of the third embodiment of the method for determining the posture of a display screen control device of the present application.

[0100] In this embodiment, before receiving the control device angle acquired by the control device sensor in step S10 and acquiring the display screen angle through the display screen sensor, steps S11 to S14 are further included:

[0101] Step S11: obtaining a virtual scene to be displayed;

[0102] Step S12: performing image rendering on the virtual scene through a rendering camera to generate a left-eye rendered image and a right-eye rendered image of the virtual scene;

[0103] Step S13: determining the observation position of the human eye, and performing parallax analysis of the observation position of the human eye, combining the left-eye rendered image and the right-eye rendered image, to perform image restoration and display on a display screen.

[0104] In this embodiment, the display screen supports the display of virtual scenes in a naked-eye 3D manner. Based on the fact that the images seen by the left and right eyes are not exactly the same from different perspectives, the display screen control system can render a left-eye rendered image and a right-eye rendered image separately based on the virtual scene and the corresponding display angle. The display screen control system uses a rendering camera to simulate the left eye's and right eye's sight lines to render the images, generating a left-eye rendered image from the left eye's perspective and a right-eye rendered image from the right eye's perspective.

[0105] Furthermore, the display screen will determine the user's eye position through methods such as collecting facial images through sensors or using glasses' light recognition. This eye position is the spatial coordinate of the eyeball in the display screen's coordinate system. The display screen will further perform parallax analysis on the eye position to determine how to combine the left-eye rendered image and the right-eye rendered image to restore the rendered image. The combined rendered image is then displayed on the display screen, allowing the user's left and right eyes to see different images when viewing the display screen, thereby creating a three-dimensional perception and a naked-eye 3D effect.

[0106] In an embodiment of the present application, the display screen renders the left-eye rendering image and the right-eye rendering image respectively, performs parallax analysis on the rendered image of the virtual scene through the parallax of the two eyes, and combines the left-eye rendering image and the right-eye rendering image to restore and display them, thereby forming a naked-eye 3D effect.

[0107] Based on the same inventive concept, this application also provides a fourth embodiment, referring to Figure 4 , Figure 4 This is a flow chart of the fourth embodiment of the method for determining the posture of a display screen control device of the present application.

[0108] In this embodiment, the method for determining the posture of the display screen control device further includes steps S51-S52:

[0109] Step S51: updating the display screen posture information of the display screen in real time according to the display screen angle;

[0110] Step S52: Calculating the angle difference between the control device and the display screen based on the updated display screen posture information and the control device angle.

[0111] In this embodiment, in order to reduce the amount of calculation based on the posture of the display screen control system, the display screen control system can correct the display screen posture and / or the display screen coordinate system based on the display screen posture at preset time intervals.

[0112] In one embodiment, a display screen control system can track and locate the control device based on changes in the device's position in space. Specifically, the display screen control system can determine the control device's position based solely on the control device's angles and the coordinates of the positioning point, even when determining the display screen's position or the angles of the X, Y, and Z axes of the display coordinate system. This reduces the amount of continuous calculation required for the control device's position during tracking.

[0113] Specifically, the display control system uses the display sensor to detect changes in the display angle in real time. When the display angle changes, it triggers an update of the display's position information. The display angle is obtained, and the current display position information is calculated based on the display angle. This information replaces the currently stored display position information, updating the position information and / or the X, Y, and Z axis angles of the display coordinate system. If the display angle remains unchanged, the angle difference and the control device's position are calculated using the original display position and / or display angle.

[0114] Optionally, the display control system can also be based on angle sensing devices such as gyroscopes. When the angle information changes, it triggers the update process of the display posture information, and updates the display posture information based on the changed angle information, or the angles of the X, Y, and Z axes in the display coordinate system.

[0115] The present application provides a posture determination device for a display screen control device, the device comprising: at least one processor; and a memory communicatively connected to the at least one processor; wherein the memory stores instructions executable by the at least one processor, and the instructions are executed by the at least one processor so that the at least one processor can execute the posture determination method for the display screen control device in the above-mentioned embodiment 1.

[0116] Reference below Figure 5 , which shows a schematic structural diagram of a device for determining the posture of a display screen control device suitable for implementing an embodiment of the present application. The device for determining the posture of the display screen control device in the embodiment of the present application may include, but is not limited to, mobile terminals such as mobile phones, laptop computers, digital broadcast receivers, PDAs (Personal Digital Assistants), PADs (Portable Application Descriptions), PMPs (Portable Media Players), in-vehicle terminals (e.g., in-vehicle navigation terminals), and fixed terminals such as digital TVs and desktop computers. Figure 5 The posture determination device of the display screen control device shown is merely an example and should not bring any limitation to the functions and scope of use of the embodiments of the present application.

[0117] like Figure 5As shown, the posture determination device of the display screen control device may include a processing device 1001 (e.g., a core processor, graphics processor, etc.), which can perform various appropriate actions and processes based on programs stored in a read-only memory (ROM) 1002 or programs loaded from a storage device 1003 into a random access memory (RAM) 1004. RAM 1004 also stores various programs and data required for the operation of the posture determination device of the display screen control device. Processing device 1001, read-only memory (ROM) 1002, and random access memory (RAM) 1004 are interconnected via a bus 1005. An input / output (I / O) interface 1006 is also connected to the bus. Typically, the following systems can be connected to I / O interface 1006: input device 1007 including, for example, a touch screen, touchpad, keyboard, mouse, image sensor, microphone, accelerometer, gyroscope, etc.; output device 1008 including, for example, a liquid crystal display (LCD), speaker, vibrator, etc.; storage device 1003 including, for example, a magnetic tape, hard disk, etc.; and communication device 1009. Communication device 1009 can allow the posture determination device of the display screen control device to communicate wirelessly or wired with other devices to exchange data. Although the figure shows the posture determination device of the display screen control device with various systems, it should be understood that implementation or presence of all the illustrated systems is not required. More or fewer systems may be implemented or present instead.

[0118] In particular, according to the embodiments disclosed herein, the processes described above with reference to the flowcharts can be implemented as computer software programs. For example, the embodiments disclosed herein include a computer program product comprising a computer program carried on a computer-readable medium, the computer program containing program code for executing the methods illustrated in the flowcharts. In such embodiments, the computer program can be downloaded and installed from a network via a communication device, or installed from storage device 1003, or installed from read-only memory (ROM) 1002. When the computer program is executed by processing device 1001, the functions defined in the methods of the embodiments disclosed herein are performed.

[0119] The posture determination device for a display screen control device provided in this application, employing the posture determination method for a display screen control device in the aforementioned embodiment, can resolve the technical issue of insufficient accuracy in the posture calibration results of a control pen using the angle sensor in a VR display. Compared to the prior art, the beneficial effects of the posture determination device for a display screen control device provided in this application are the same as those of the posture determination method for a display screen control device provided in the aforementioned embodiment. Other technical features of the posture determination device for a display screen control device are the same as those disclosed in the method of the aforementioned embodiment and are not further elaborated upon here.

[0120] It should be understood that the various parts disclosed in this application can be implemented using hardware, software, firmware, or a combination thereof. In the description of the above embodiments, specific features, structures, materials, or characteristics can be combined in any one or more embodiments or examples in a suitable manner.

[0121] The above description is merely a specific embodiment of the present application, but the scope of protection of the present application is not limited thereto. Any changes or substitutions that can be easily conceived by a person skilled in the art within the technical scope disclosed in this application should be included in the scope of protection of this application. Therefore, the scope of protection of this application should be based on the scope of protection of the claims.

[0122] The present application provides a computer-readable storage medium having computer-readable program instructions (ie, a computer program) stored thereon, and the computer-readable program instructions are used to execute the posture determination method of the display screen control device in the above-mentioned embodiment.

[0123] The computer-readable storage medium provided herein may be, for example, a USB flash drive, but is not limited to electrical, magnetic, optical, electromagnetic, infrared, or semiconductor systems, systems, or devices, or any combination thereof. More specific examples of computer-readable storage media may include, but are not limited to, an electrical connection having one or more wires, a portable computer disk, a hard disk, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination thereof. In this embodiment, the computer-readable storage medium may be any tangible medium that contains or stores a program that can be used by or in conjunction with an instruction execution system, system, or device. The program code contained on the computer-readable storage medium may be transmitted using any suitable medium, including, but not limited to, wires, optical cables, RF (Radio Frequency), etc., or any suitable combination thereof.

[0124] The computer-readable storage medium may be included in the posture determination device of the display screen control device; or may exist independently without being assembled into the posture determination device of the display screen control device.

[0125] The above-mentioned computer-readable storage medium carries one or more programs. When the above-mentioned one or more programs are executed by the posture determination device of the display screen control device, the posture determination device of the display screen control device: receives the control device angle collected by the control device sensor, and obtains the display screen angle through the display screen sensor; calculates the angle difference between the control device angle and the display screen angle; identifies the positioning point coordinates of the control device through the positioning sensor, and determines the posture information of the control device based on the angle difference and the positioning point coordinates.

[0126] Computer program code for performing the operations of the present application may be written in one or more programming languages, or a combination thereof, including object-oriented programming languages ​​such as Java, Smalltalk, C++, and conventional procedural programming languages ​​such as "C" or similar programming languages. The program code may be executed entirely on the user's computer, partially on the user's computer, as a stand-alone software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In the case of a remote computer, the remote computer may be connected to the user's computer via any type of network, including a local area network (LAN) or a wide area network (WAN), or may be connected to an external computer (e.g., via the Internet using an Internet service provider).

[0127] The flow charts and block diagrams in the accompanying drawings illustrate the possible architecture, functions and operations of the systems, methods and computer program products according to various embodiments of the present application. In this regard, each box in the flow chart or block diagram can represent a module, program segment or a part of code, and the module, program segment or a part of code contains one or more executable instructions for realizing the specified logical function. It should also be noted that in some alternative implementations, the functions marked in the box can also occur in a different order than that marked in the accompanying drawings. For example, two boxes represented in succession can actually be executed substantially in parallel, and they can sometimes be executed in the opposite order, depending on the functions involved. It should also be noted that each box in the block diagram and / or flow chart, and the combination of the boxes in the block diagram and / or flow chart can be implemented by a dedicated hardware-based system that performs the specified function or operation, or can be implemented by a combination of dedicated hardware and computer instructions.

[0128] The modules described in the embodiments of the present application may be implemented in software or hardware, wherein the name of a module does not necessarily limit the unit itself.

[0129] The computer-readable storage medium provided in this application stores computer-readable program instructions (i.e., a computer program) for executing the aforementioned method for determining the posture of a display screen control device. This computer-readable storage medium can address the technical issue of insufficient accuracy in the posture calibration results of a VR display screen using the angle sensor in the control pen. Compared to the prior art, the beneficial effects of the computer-readable storage medium provided in this application are similar to those of the method for determining the posture of a display screen control device provided in the aforementioned embodiments, and are not further elaborated here.

[0130] The above description is only part of the embodiments of the present application and does not limit the patent scope of the present application. All equivalent structural transformations made by using the contents of the present application specification and drawings under the technical concept of the present application, or direct / indirect application in other related technical fields are included in the patent protection scope of the present application.

Claims

1. A method for determining the posture of a display screen control device, characterized in that: The method for determining the posture of the display screen control device comprises the following steps: Receiving a control device angle acquired by a control device sensor, and acquiring a display screen angle through a display screen sensor, wherein the control device sensor and / or the display screen sensor include a geomagnetic sensor and a gyroscope sensor; Calculating the angle difference between the control device angle and the display screen angle; receiving a positioning signal of the control device through a positioning sensor; collecting a positioning image of the control device by at least two optical sensors; Identify a positioning recognition light spot of the control device based on the positioning point in the positioning image; Determining the coordinates of the positioning point by triangulation according to the position of the positioning recognition light spot mapped to the positioning image and the sensor coordinates of the light sensor in the display screen coordinate system; Determining the position information of the control device in the display screen coordinate system according to the positioning point coordinates and the angle difference; According to the posture information, a control instruction of the control device is determined, and an interactive action corresponding to the control instruction is implemented in the virtual scene.

2. The method according to claim 1, wherein The step of determining a control instruction of a control device according to the posture information and implementing an interactive action corresponding to the control instruction in a virtual scene includes: receiving a control signal from the control device; Determining a control instruction of the control device according to the posture information and the control signal; Determining an interactive action corresponding to the control instruction in the virtual scene; The interactive action is executed, and the interactive action and the interactive result of the interactive action in the virtual scene are displayed on a display screen.

3. The method according to claim 2, wherein After the step of identifying the positioning point coordinates of the control device by a positioning sensor and determining the posture information of the control device according to the angle difference and the positioning point coordinates, the method further includes: determining an extension line of the control device according to the relative posture of the control device and the display screen; determining a target extension line of the extension line in the virtual scene, and using the target extension line as a control line of the virtual scene; The control line is displayed in the virtual scene displayed on the display screen.

4. The method according to claim 1, wherein The step of determining the position information of the control device in the display screen coordinate system according to the positioning point coordinates and the angle difference includes: Determining, in the display screen coordinate system, posture information of the control device based on the angle difference; and, determining the position information of the control device according to the coordinates of the positioning point and the shape of the control device; The posture information is generated based on the position information and the posture information.

5. The method according to claim 1, wherein Before the steps of receiving the control device angle acquired by the control device sensor and acquiring the display screen angle through the display screen sensor, the method further includes: Obtaining a virtual scene to be displayed; Performing image rendering on the virtual scene by using a rendering camera to generate a left-eye rendered image and a right-eye rendered image of the virtual scene; Determine the human eye observation position, and through the parallax analysis of the human eye observation position, combine the left eye rendering image and The right eye rendered image is restored and displayed on a display screen.

6. The method according to claim 1, wherein The step of calculating the angle difference between the control device angle and the display screen angle further comprises: According to the display screen angle, the display screen posture information of the display screen is updated in real time; The angle difference between the control device and the display screen is calculated based on the updated display screen posture information and the control device angle.

7. A device for determining the posture of a display screen control device, characterized in that: The device includes: a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the computer program is configured to implement the steps of the method for determining the posture of a display screen control device according to any one of claims 1 to 6.

8. A storage medium, characterized in that: The storage medium is a computer-readable storage medium, and a computer program is stored on the storage medium. When the computer program is executed by a processor, the steps of the method for determining the posture of a display screen control device according to any one of claims 1 to 6 are implemented.

Images