Three-dimensional interactive display method and electronic equipment

Abstract

The invention provides a three-dimensional interactive display method and electronic equipment, and relates to the technical field of terminals. According to the method and the device, the three-dimensional display capability of the first electronic equipment is combined, and through interaction with the second electronic equipment, three-dimensional display and interaction experience can be provided for a user. Wherein the first electronic equipment is fixedly placed at a first position, and the method comprises the following steps: the first electronic equipment acquires a first positioning result of second electronic equipment, and the first positioning result is determined through a wireless positioning signal. Afterwards, the first electronic equipment can display a three-dimensional picture, the three-dimensional picture comprises a first virtual element matched with the first positioning result, and the three-dimensional picture is used for at least one user to watch or interact outside the first electronic equipment.

Description

Technical Field

[0001] The present application relates to the field of terminal technology, and in particular to a three-dimensional interactive display method and electronic device. Background Art

[0002] With the development of terminal technology, some electronic devices can provide users with three-dimensional (3D) display experience. For example, smart screens support 3D display, and users can use smart screens to watch 3D movies. However, smart screens cannot provide users with a 3D interactive experience. For another example, VR games developed based on virtual reality (VR) technology, users can watch the 3D game interface by wearing a VR headset, and can interact with 3D game elements through a controller that works with the VR headset. However, the weight of the VR headset is heavy and can easily cause user fatigue, and users cannot observe the real environment around them after wearing a 3D headset, which can easily cause danger. Summary of the invention

[0003] In order to solve the above technical problems, the present application provides a three-dimensional interactive display method and an electronic device. The technical solution provided by the present application provides a user with a three-dimensional display and interactive experience by combining the three-dimensional display capability of a first electronic device and interacting with a second electronic device. In addition, the user can view the three-dimensional image outside the first electronic device, reducing the burden on the user to wear the device.

[0004] In order to achieve the above technical objectives, this application provides the following technical solutions:

[0005] In a first aspect, a three-dimensional interactive display method is provided, which is applied to a first electronic device. The first electronic device is fixedly placed at a first position, and the method includes: obtaining a first positioning result of a second electronic device, where the first positioning result is determined by a wireless positioning signal. A three-dimensional picture is displayed, where the three-dimensional picture includes a first virtual element matching the first positioning result, and the three-dimensional picture is used for at least one user to watch or interact with outside the first electronic device.

[0006] In some examples, the first electronic device is fixedly placed at a first position, a spatial coordinate system is established based on the position of the first electronic device, and the three-dimensional coordinates and orientation information of the second electronic device relative to the first electronic device are obtained. For example, the first positioning result of the second electronic device includes the three-dimensional coordinates and orientation information.

[0007] In this way, the first electronic device includes a first virtual element matching the first positioning result in the displayed three-dimensional picture according to the first positioning result of the second electronic device, thereby realizing the interaction between the user and the three-dimensional picture displayed by the first electronic device. For example, the moving direction of the first virtual element is related to the pointing information of the second electronic device. For another example, the display position of the first virtual element in the three-dimensional picture is related to the moving position of the second electronic device operated by the user.

[0008] Furthermore, the user can view the 3D image outside the first electronic device, thereby preventing the user's sight from being completely blocked. The user can view the environment in the real world while viewing the 3D image, thereby improving the user's safety.

[0009] In some examples, the three-dimensional picture includes an image picture, a video picture, an application display picture, a picture that can interact with a user, etc. For example, the three-dimensional picture includes multiple types such as a static image or a frame image in a video picture.

[0010] According to the first aspect, or any implementation of the first aspect above, the method further includes: acquiring a second positioning result of the third electronic device, and the three-dimensional picture also includes a second virtual element matching the second positioning result.

[0011] According to a first aspect, a three-dimensional picture is provided for simultaneous viewing by a plurality of users.

[0012] According to the first aspect, or any implementation of the first aspect, the second electronic device and the third electronic device are operated by one or two users.

[0013] In some examples, the second electronic device and the third electronic device are operated by one or two users. For example, the second electronic device and the third electronic device are a pair of remote control handles, and the user can operate the two remote control handles with the left and right hands respectively. Then, the first virtual element and the second virtual element are the same virtual elements corresponding to the user. For another example, the second electronic device and the third electronic device are a pair of remote control handles, and two users operate the two remote control handles respectively. Then, the first virtual element and the second virtual element are different virtual elements corresponding to the two users respectively. It should be understood that the first electronic device can also be connected to more devices, so that more users can interact with the same three-dimensional picture displayed by the first electronic device by operating different devices. For example, the user can configure a corresponding number of remote control handles according to needs.

[0014] In this way, multiple users can watch the same 3D picture by wearing 3D glasses or with naked eyes, thereby sharing the same picture, and can operate the same VR software through different devices to achieve a multi-person interactive 3D virtual world experience.

[0015] Moreover, in multi-person 3D interactive scenarios, users do not need to purchase and wear multiple VR headsets, which reduces costs while providing the same user experience. And there are no disadvantages in using VR headsets.

[0016] According to the first aspect, or any implementation of the first aspect, the first virtual element is at least one of a display component, a controllable component, and a trajectory matching the movement of the second electronic device.

[0017] For example, the first electronic device displays the first virtual element, that is, the first virtual element is a display component. For another example, the first electronic device controls the change of the first virtual element in the three-dimensional picture, such as the change of the display position, size, shape, etc., in response to the user's operation on the second electronic device, that is, the first virtual element is a controllable component. For another example, the first electronic device displays the movement track of the first virtual element in the process of moving the display position of the first virtual element in the three-dimensional picture in response to the user's operation on the second electronic device, that is, the first virtual element is a track that matches the movement of the second electronic device.

[0018] According to the first aspect, or any implementation of the first aspect above, the method further includes: receiving input data sent by the second electronic device, the input data including at least one of key input data, input data corresponding to hand movements, and input data corresponding to joystick operations. The input data is used to interact with the three-dimensional picture.

[0019] According to the first aspect, or any implementation of the first aspect above, the input data corresponding to the joystick operation is used to instruct to update the display content or display angle of the three-dimensional picture.

[0020] In this way, by converting the front and back or left and right inputs of the joystick through VR software, it can be equivalent to the 6DOF detection of the VR headset. When the user's head does not move, the screen perspective can be changed only by the joystick, thus avoiding the user's dizziness caused by low frame or frame loss.

[0021] In addition, the user's perspective is adjusted based on the joystick movement, and there is no need to generate reverse motion images, thereby reducing system resource usage, lowering processor requirements, and reducing the possibility of frame drops. In addition, the perspective of the image changes smoothly.

[0022] In some examples, the second electronic device is a device equipped with a display screen, such as a mobile phone, a tablet, etc. Then, the second electronic device can receive user operations by displaying virtual buttons and generate corresponding input data. For example, the second electronic device detects the user's operation of the up, down, left, and right controls displayed on the display screen, and generates corresponding input data, which can also be converted into the pitch or tilt motion data of the user's head. Thus, the second electronic device enables the user to interact with the three-dimensional picture displayed by the first electronic device through the display of virtual buttons. In some examples, the second electronic device determines the direction and position of the user moving the second electronic device through sensor data, and can generate corresponding positioning results.

[0023] According to the first aspect, or any implementation of the first aspect above, the method further includes: obtaining a third positioning result of the second electronic device, and updating the display content or display angle of the three-dimensional image according to the third positioning result.

[0024] In this way, in response to the change of the positioning result of the second electronic device, the first electronic device can also update the display of the three-dimensional picture accordingly, thereby realizing the interaction between the user and the three-dimensional picture.

[0025] According to the first aspect, or any implementation of the first aspect above, before displaying the three-dimensional picture, the method also includes: determining the three-dimensional picture through a first software in a first electronic device according to the first positioning result, and the first software is a first virtual reality VR software.

[0026] In this way, after the first electronic device generates a three-dimensional image through VR software, it can be sent directly to the display module for display. There is no need for the image processing process of the VR headset to go through the deformation algorithm and the head tracking algorithm, thereby reducing the requirements for computing power and reducing the image processing delay, avoiding the display frame rate reduction or abnormal freezing.

[0027] According to the first aspect, or any implementation of the first aspect above, before displaying the three-dimensional picture, the method further includes: sending the first positioning result to the fourth electronic device. Receiving the three-dimensional picture fed back by the fourth electronic device, the three-dimensional picture is obtained by the fourth electronic device after processing the first positioning result through the second software in the fourth electronic device, and the second software is the second VR software.

[0028] In this way, by configuring a fourth electronic device with stronger external computing power to obtain a 3D image, the computing power requirement for the first electronic device is reduced. The first electronic device also directly displays the 3D image after obtaining it, without having to go through the deformation algorithm and head tracking algorithm, which has lower requirements for delay and resources, and further reduces the risk of lag.

[0029] According to the first aspect, or any implementation of the first aspect above, after obtaining the first positioning result of the second electronic device, the method also includes: collecting user images through a camera, and the user images are used to determine at least one parameter of the number of users, user locations, and user physical information.

[0030] In this way, the initialization of the first electronic device is completed through image acquisition of the first electronic device, and the user does not need to participate in the initialization every time after starting the device, thereby reducing the difficulty of user operation while ensuring the accurate effect of three-dimensional display.

[0031] According to the first aspect, or any implementation of the first aspect above, before obtaining the first positioning result of the second electronic device, the method further includes: obtaining location information of the first electronic device, where the location information is used for displaying a three-dimensional picture.

[0032] For example, when the first electronic device starts the VR software in the first electronic device for the first time, it obtains its own position information to initialize the VR software and adjust the viewing angle height of the user using the second electronic device in the three-dimensional picture. Optionally, the first electronic device can save the position information, so that the display of the three-dimensional picture can be adjusted according to the position information during the subsequent three-dimensional picture display process.

[0033] In this way, compared with VR head displays, which require users to manually perform initialization calibration after each startup and determine the safe zone before use, the three-dimensional interactive display method provided in the embodiment of the present application generally does not need to move the first electronic device. Based on the storage and display characteristics of the smart screen, it is not necessary to force the user to perform the initialization calibration process after startup, which effectively simplifies user operations and reduces the difficulty of user use.

[0034] According to the first aspect, or any implementation of the first aspect above, the first positioning result includes at least one of the distance between the second electronic device and the first electronic device, the orientation information of the second electronic device, and the height information of the second electronic device.

[0035] According to the first aspect, or any implementation of the first aspect above, a first positioning module is configured in the first electronic device, and a second positioning module is configured in the second electronic device, and obtaining a first positioning result of the second electronic device includes: sending a first wireless positioning signal to the second electronic device through the first positioning module. Receiving the first positioning result sent by the second electronic device, the first positioning result is determined by the second electronic device through the second positioning module according to the first wireless positioning signal. And / or, receiving the second wireless positioning signal sent by the second electronic device through the second positioning module through the first positioning module. Determining the first positioning result through the first positioning module according to the second wireless positioning signal.

[0036] In this way, the first electronic device can achieve more accurate positioning of the second electronic device through the wireless positioning signal, so as to subsequently display a three-dimensional image that better meets the user's needs.

[0037] According to the first aspect, or any implementation of the first aspect above, the first positioning module or the second positioning module is any one of the following: an ultra-wideband technology UWB positioning module, an electromagnetic positioning module, a millimeter wave positioning module, and an optical tracking positioning module.

[0038] According to the first aspect, or any implementation of the first aspect above, the three-dimensional image is used for at least one user to watch outside the first electronic device, including: the three-dimensional image is used for at least one user to watch wearing three-dimensional glasses or watching with naked eyes, and the three-dimensional glasses include shutter glasses or polarized glasses.

[0039] In this way, users can watch 3D images and interact with them by wearing 3D glasses. Compared with VR headsets, 3D glasses are much lighter and can provide the same immersive virtual world interactive experience as VR headsets. In addition, 3D glasses do not require external power supply, which will not cause discomfort such as heating and stuffiness, thus effectively improving user comfort.

[0040] In addition, when users wear 3D glasses or watch 3D images with naked eyes, their vision will not be blocked and they can observe the real environment around them at the same time, thereby effectively improving their safety.

[0041] In some embodiments, the three-dimensional interactive display method provided in the embodiments of the present application can also be applied to a spliced ​​screen scenario, wherein the first electronic device is a spliced ​​screen device, the display screen of the first electronic device includes multiple spliced ​​screens, and the first electronic device displays a spliced ​​three-dimensional image according to the positioning result of the second electronic device.

[0042] In some examples, the user can interact with the spliced ​​three-dimensional picture displayed by the first electronic device through the second electronic device. For example, the user controls the first virtual element to move from one spliced ​​screen to another spliced ​​screen for display by moving the second electronic device. For another example, when the user controls the movement of the first virtual element through the second electronic device, the movement track covers multiple spliced ​​screens, and the first electronic device displays the spliced ​​movement track on multiple spliced ​​screens.

[0043] In some embodiments, the three-dimensional interactive display method provided in the embodiments of the present application can also be applied to multi-screen collaborative scenarios. Among them, the first electronic device can establish a communication connection with other electronic devices with display functions and realize display interaction. For example, the first electronic device is electronic device 1, and the electronic device that performs display interaction with the first electronic device is electronic device 2. Among them, electronic device 1 supports 3D display, and electronic device 2 supports 2D display or 3D display.

[0044] In some examples, while electronic device 1 is displaying a three-dimensional picture 1, in response to a user moving a second electronic device toward electronic device 2 to indicate an operation of displaying the first virtual element on electronic device 2, electronic device 1 may send display information of the three-dimensional picture 1 to electronic device 2.

[0045] When the electronic device 2 supports 3D display, the electronic device 2 can display the three-dimensional picture 2 according to the display information of the three-dimensional picture 1, and the three-dimensional picture 2 includes the first virtual element. Among them, the three-dimensional picture 2 is the same as or corresponds to the three-dimensional picture 1, so as to realize the continuous display of the display picture. In this way, the electronic device 1 realizes the three-dimensional picture interaction between different electronic devices according to the user operation, and enriches the user's interaction needs. For example, during the movement of the first virtual element, the electronic device 2 is being operated by another user, so that the three-dimensional picture 2 displayed by the electronic device 2 may include the first virtual element and another virtual element corresponding to the other user.

[0046] When the electronic device 2 supports 2D display, the electronic device 2 can display a two-dimensional picture according to the display information of the three-dimensional picture 1, and the two-dimensional picture includes a two-dimensional virtual element corresponding to the first virtual element. The two-dimensional picture corresponds to the three-dimensional picture 1. In this way, the electronic device 1 realizes the interaction of display pictures between different electronic devices according to the user operation, and further enriches the user's interaction needs.

[0047] In other examples, electronic device 2 displays a two-dimensional screen, which includes a two-dimensional virtual element corresponding to user 1. In response to the user operation, electronic device 2 determines to move the two-dimensional virtual element to electronic device 1 for display, and electronic device 2 may send display information of the two-dimensional screen to electronic device 1. Correspondingly, after receiving the display information of the two-dimensional screen, electronic device 1 may display a corresponding three-dimensional screen according to the display information, and the three-dimensional screen includes a three-dimensional virtual element corresponding to the two-dimensional virtual element.

[0048] In this way, when the electronic device supports 2D display and / or 3D display, different electronic devices can respond to user operations to achieve interaction of display images, thereby enriching the interaction scenarios and meeting the user's usage needs.

[0049] It should be understood that a multi-screen collaboration scenario may include two or more electronic devices, which may support 2D display or 3D display, or 4D or higher display. In response to the user's operation on the second electronic device (such as a remote control handle, etc.), different electronic devices can display corresponding display screens through the interaction of display information to meet the user's interaction needs. This embodiment of the application will not be exemplified one by one.

[0050] In a second aspect, a first electronic device is provided. The first electronic device includes: a processor, a memory and a display screen, the memory and the display screen are coupled to the processor, the memory is used to store computer program code, the computer program code includes computer instructions, when the processor reads the computer instructions from the memory, the first electronic device executes: obtain a first positioning result of the second electronic device, the first positioning result is determined by a wireless positioning signal. Display a three-dimensional picture, the three-dimensional picture includes a first virtual element matching the first positioning result, the three-dimensional picture is used for at least one user to watch or interact outside the first electronic device. Wherein, the first electronic device is fixedly placed in the first position.

[0051] According to a second aspect, a three-dimensional picture is provided for simultaneous viewing by a plurality of users.

[0052] According to the second aspect, or any implementation of the second aspect above, when the processor reads computer instructions from the memory, it also enables the first electronic device to execute: obtaining a second positioning result of the third electronic device, and the three-dimensional picture also includes a second virtual element matching the second positioning result.

[0053] According to the second aspect, or any implementation of the second aspect, the second electronic device and the third electronic device are operated by one or two users.

[0054] According to the second aspect, or any implementation of the second aspect, the first virtual element is at least one of a display component, a controllable component, and a trajectory matching the movement of the second electronic device.

[0055] According to the second aspect, or any implementation of the second aspect above, when the processor reads the computer instruction from the memory, the first electronic device is further caused to execute: receiving input data sent by the second electronic device, the input data including at least one of key input data, input data corresponding to hand movements, and input data corresponding to joystick operations. The input data is used to interact with the three-dimensional picture.

[0056] According to the second aspect, or any implementation of the second aspect, the input data corresponding to the joystick operation is used to instruct to update the display content or display angle of the three-dimensional picture.

[0057] According to the second aspect, or any implementation of the second aspect above, when the processor reads the computer instruction from the memory, the first electronic device is also caused to execute: obtaining a third positioning result of the second electronic device. According to the third positioning result, the display content or display angle of the three-dimensional image is updated.

[0058] According to the second aspect, or any implementation of the second aspect above, when the processor reads computer instructions from the memory, it also enables the first electronic device to execute: based on the first positioning result, determine the three-dimensional picture through the first software in the first electronic device, and the first software is the first virtual reality VR software.

[0059] According to the second aspect, or any implementation of the second aspect above, when the processor reads the computer instruction from the memory, the first electronic device is also caused to execute: sending the first positioning result to the fourth electronic device. Receive a three-dimensional picture fed back by the fourth electronic device, the three-dimensional picture is obtained by the fourth electronic device after processing the first positioning result through the second software in the fourth electronic device, and the second software is the second VR software.

[0060] According to the second aspect, or any implementation of the second aspect above, when the processor reads computer instructions from the memory, it also enables the first electronic device to execute: collecting user images through a camera, and the user images are used to determine at least one parameter of the number of users, user locations, and user physical information.

[0061] According to the second aspect, or any implementation of the second aspect above, when the processor reads the computer instruction from the memory, it also enables the first electronic device to execute: obtaining location information of the first electronic device, and the location information is used for displaying a three-dimensional picture.

[0062] According to the second aspect, or any implementation of the second aspect above, the first positioning result includes at least one of the distance between the second electronic device and the first electronic device, the orientation information of the second electronic device, and the height information of the second electronic device.

[0063] According to the second aspect, or any implementation of the second aspect above, a first positioning module is configured in the first electronic device, and a second positioning module is configured in the second electronic device, and obtaining a first positioning result of the second electronic device includes: sending a first wireless positioning signal to the second electronic device through the first positioning module. Receiving the first positioning result sent by the second electronic device, the first positioning result is determined by the second electronic device through the second positioning module according to the first wireless positioning signal. And / or, receiving the second wireless positioning signal sent by the second electronic device through the second positioning module through the first positioning module. Determining the first positioning result through the first positioning module according to the second wireless positioning signal.

[0064] According to the second aspect, or any implementation of the second aspect above, the first positioning module or the second positioning module is any one of the following: an ultra-wideband technology UWB positioning module, an electromagnetic positioning module, a millimeter wave positioning module, and an optical tracking positioning module.

[0065] According to the second aspect, or any implementation of the second aspect above, the three-dimensional image is used for at least one user to watch outside the first electronic device, including: the three-dimensional image is used for at least one user to watch wearing three-dimensional glasses or watching with naked eyes, and the three-dimensional glasses include shutter glasses or polarized glasses.

[0066] In a third aspect, an electronic device is provided, which has the function of implementing the three-dimensional interactive display method described in the first aspect and any possible implementation thereof. The function can be implemented by hardware, or by hardware executing corresponding software. The hardware or software includes one or more modules corresponding to the above functions.

[0067] In a fourth aspect, a computer-readable storage medium is provided. The computer-readable storage medium stores a computer program (also referred to as an instruction or code), and when the computer program is executed by an electronic device, the electronic device executes the method of the first aspect or any one of the implementations of the first aspect.

[0068] According to a fifth aspect, a computer program product is provided. When the computer program product is run on an electronic device, the electronic device executes the method according to the first aspect or any one of the implementations of the first aspect.

[0069] According to a sixth aspect, a circuit system is provided, the circuit system comprising a processing circuit, wherein the processing circuit is configured to execute the method according to the first aspect or any one of the embodiments of the first aspect.

[0070] In the seventh aspect, a chip system is provided, comprising at least one processor and at least one interface circuit, wherein the at least one interface circuit is used to perform transceiver functions and send instructions to the at least one processor, and when the at least one processor executes the instructions, the at least one processor executes the method of the first aspect or any one of the embodiments of the first aspect.

[0071] The technical effects of the aforementioned aspects can be referenced to each other and will not be repeated here. BRIEF DESCRIPTION OF THE DRAWINGS

[0072] Figure 1 Schematic diagram of the display scene of the VR head display provided in the embodiment of the present application Figure 1 ;

[0073] Figure 2 Schematic diagram of the display scene of the VR head display provided in the embodiment of the present application Figure 2 ;

[0074] Figure 3 A schematic diagram of a communication system for applying the three-dimensional interactive display method provided in an embodiment of the present application;

[0075] Figure 4A schematic diagram of the hardware structure of a first electronic device provided in an embodiment of the present application;

[0076] Figure 5 Schematic diagram of the 3D interactive display scene provided in the embodiment of the present application Figure 1 ;

[0077] Figure 6 Schematic diagram of the 3D interactive display scene provided in the embodiment of the present application Figure 2 ;

[0078] Figure 7 Schematic diagram of the 3D interactive display scene provided in the embodiment of the present application Figure 3 ;

[0079] Figure 8 Schematic diagram of the 3D interactive display scene provided in the embodiment of the present application Figure 4 ;

[0080] Fig. 9 A schematic diagram of the joystick control principle provided in the embodiment of the present application;

[0081] Fig.10 Schematic diagram of the 3D interactive display scene provided in the embodiment of the present application Figure 5 ;

[0082] Fig.11 A schematic diagram of a flow chart of a three-dimensional interactive display method provided in an embodiment of the present application;

[0083] Fig.12 A schematic diagram of the structure of a first electronic device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0084] The technical solutions in the embodiments of the present application are described below in conjunction with the accompanying drawings in the embodiments of the present application. Wherein, in the description of the embodiments of the present application, the terms used in the following embodiments are only for the purpose of describing specific embodiments, and are not intended to be used as limitations on the present application. As used in the specification and the appended claims of the present application, the singular expressions "one", "a kind of", "said", "above", "the" and "this" are intended to include expressions such as "one or more", unless there is a clear indication to the contrary in the context. It should also be understood that in the following embodiments of the present application, "at least one", "one or more" refer to one or more (including two).

[0085] References to "one embodiment" or "some embodiments" etc. described in this specification mean that one or more embodiments of the present application include specific features, structures or characteristics described in conjunction with the embodiment. Thus, the statements "in one embodiment", "in some embodiments", "in some other embodiments", "in some other embodiments", etc. that appear in different places in this specification do not necessarily refer to the same embodiment, but mean "one or more but not all embodiments", unless otherwise specifically emphasized in other ways. The terms "including", "comprising", "having" and their variations all mean "including but not limited to", unless otherwise specifically emphasized in other ways. The term "connection" includes direct connection and indirect connection, unless otherwise specified. "First" and "second" are used for descriptive purposes only and cannot be understood as indicating or implying relative importance or implicitly indicating the number of technical features indicated.

[0086] In the embodiments of the present application, the words "exemplarily" or "for example" are used to indicate examples, illustrations or explanations. Any embodiment or design described as "exemplarily" or "for example" in the embodiments of the present application should not be interpreted as being more preferred or more advantageous than other embodiments or designs. Specifically, the use of words such as "exemplarily" or "for example" is intended to present related concepts in a specific way.

[0087] In some embodiments, based on virtual display technology, VR devices can put users into a virtual world, enabling users to interact with the virtual world in a virtual environment. For example, by wearing a VR head display (or helmet, etc.), users can be put into a virtual 3D game environment and interact with virtual elements in the 3D game, thereby enhancing the user's gaming experience.

[0088] In some examples, the VR device includes a VR headset, a controller, and a sensor. Among them, after the user wears the VR headset, the VR headset can be used to provide the user with a 3D picture display and sound playback experience separated for the left and right eyes, so that the user can immerse in the virtual world. The controller is, for example, a handle, a glove, etc., and the user can interact with the virtual world through the controller. For example, the user controls the movement of virtual elements through the controller, and provides operation feedback to the user through the vibration of the controller. The sensor is used to track the user's position and action in the virtual world to achieve a more realistic interactive experience. For example, the VR headset determines the height of the VR headset relative to the ground through a height sensor, so that the height is used as the user's height in subsequent work projects. For another example, the VR headset obtains the user's head movement parameters through a gyroscope sensor and an acceleration sensor to adjust the direction and viewing angle of the 3D display screen to ensure the user's viewing experience. And through interaction with the controller, the direction of the controller and the distance between the controller and the controller can be determined.

[0089] In some examples, a VR headset is equipped with a battery to instruct the VR headset to work. However, the weight of the battery makes the VR headset heavy, which can easily cause user fatigue.

[0090] In some examples, when the VR headset is powered by an external power source, it is necessary to connect an external power cord. However, since the user cannot observe the actual environment around him while wearing the VR headset, he may be easily tripped by the power cord, which may cause danger.

[0091] In some examples, when using a VR headset, the user is completely immersed in the virtual world and cannot observe the actual environment around them, which makes it easy for the user to collide with objects in the actual environment, affecting the user's safety. For this reason, before each use of a VR headset, it is necessary to perform height and orientation calibration for the user to determine the safe zone in order to maximize the safety of the user. However, this does not completely reduce the risk of user use, and the calibration before each use also increases the difficulty of use for the user.

[0092] For example, Figure 1 As shown, during use, the directional positioning transmitting module in the VR headset sends a positioning signal to the two handles, so that the handles determine the handle positioning result according to the positioning signal, and the handle positioning result is the relative position information of the handle relative to the VR headset. Afterwards, the VR headset can obtain the positioning result sent by the handle through the wireless communication module. In addition, the VR headset can obtain the positioning result of the VR headset through a 6-degree-of-freedom (DOF) sensor. Afterwards, the VR headset combines the handle positioning result and the VR headset positioning result through VR software, and outputs the display screen based on the head tracking algorithm and the deformation algorithm, so that the output display screen can be a circular display screen that matches the left and right eye display screens, and can ensure that the picture does not move relative to the head when the user's head moves, thereby providing the user with a better virtual world display effect.

[0093] During the use of the above-mentioned VR headset, the resolution of the VR headset is limited by the resolution of the single-eye display screen. If the resolution is not high enough, it will cause a screen door effect in the human eye.

[0094] In addition, the display of VR headsets must rely on head tracking algorithms, deformation algorithms, and other processing, which require high computing power and latency. When computing power is insufficient or thermal protection is reduced, it will lead to low display frame rate or freeze, which can easily cause discomfort to users.

[0095] As another example, Figure 2As shown, a detection base station is arranged near the VR headset and the controller (such as a handle), and the detection base station is, for example, an electronic device such as a PC, a tablet, or a mobile phone. In some examples, the detection base station is configured with a wireless communication module and VR software, and is configured with a corresponding pointing and positioning transmitting module. In this way, the VR headset and the handle can be positioned by the detection base station, and a corresponding display screen can be generated according to the positioning result, and the display screen is sent to the VR headset, which is processed by the VR headset through head tracking algorithms, deformation algorithms, etc. to generate a display screen that meets the user's viewing needs.

[0096] It can be seen that in the current solution, the VR headset still has the problems existing in the VR headset mentioned above. In addition, the detection of the VR headset and the handle requires the deployment of additional detection base stations, which makes it impossible for the VR headset to be used without the detection base station and the fixed environment.

[0097] Therefore, the embodiment of the present application provides a three-dimensional interactive display method, which utilizes a display device equipped with a remote control device, combines the 3D display capability of the display device and the directional remote control capability of the remote control device, to achieve the same 3D virtual display effect as a VR headset, and does not cause the above-mentioned problems, thereby improving the user experience.

[0098] Figure 3 A schematic diagram of a communication system for the three-dimensional interactive display method provided in an embodiment of the present application. Figure 3 As shown, the communication system includes a first electronic device 100 and a second electronic device 200 .

[0099] Optionally, the first electronic device 100 may be, for example, a smart screen, a spliced ​​screen device, a tablet computer, a mobile phone, a laptop computer, an ultra-mobile personal computer (UMPC), a netbook, a personal digital assistant (PDA), an artificial intelligence (AI) device, or other terminal device with 3D display capability. The operating system installed in the first electronic device 100 includes but is not limited to: Or other operating systems. This application does not limit the specific type of the first electronic device 100 and the installed operating system.

[0100] Optionally, the second electronic device 200 may be, for example, a remote control handle, remote control gloves, wearable devices, mobile phones, AI devices, and other terminal devices with a directional remote control function. The operating system installed in the second electronic device 200 includes but is not limited to Or other operating systems, or the second electronic device 200 may not have an operating system installed. This application does not limit the specific type of the second electronic device 200, whether an operating system is installed, and the operating system installed when an operating system is installed.

[0101] In some embodiments, a wireless communication connection is established between the first electronic device 100 and the second electronic device 200. The wireless communication technology for establishing the wireless communication connection includes but is not limited to at least one of the following: ultra wide band (UWB), Bluetooth (BT) (e.g., traditional Bluetooth or low energy (BLE) Bluetooth), wireless local area networks (WLAN) (e.g., wireless fidelity (Wi-Fi) networks), near field communication (NFC), Zigbee, frequency modulation (FM), infrared (IR), etc.

[0102] Optionally, the first electronic device 100 and the second electronic device 200 may also establish a communication connection through a third-party device in the local area network, such as a router, a gateway, a smart device controller, a server, etc.

[0103] In other embodiments, a wired communication connection may also be established between the first electronic device 100 and the second electronic device 200. For example, the first electronic device 100 and the second electronic device 200 are connected via a universal serial bus (USB) interface.

[0104] Optionally, in a remote control scenario, the first electronic device 100 is, for example, a controlled display terminal (such as a smart screen), and the second electronic device 200 is, for example, a remote control terminal (such as a remote control handle). The first electronic device 100 locates the position of the second electronic device 200, combines the positioning result of the second electronic device 200 and the key signal input by the second electronic device 200, displays the corresponding 3D picture, and realizes interaction with the user. In this process, the user can watch the 3D display picture by wearing 3D glasses or naked eyes. The user can observe the real environment while entering the 3D virtual world, which improves the safety of use.

[0105] Optionally, the first electronic device 100 and the second electronic device 200 in the embodiment of the present application may be implemented by different devices, and the different devices may have the same, similar or somewhat different hardware structures, such as Figure 4The hardware structure shown.

[0106] For example, the first electronic device 100 has Figure 4 Take the hardware structure shown in the figure as an example. Figure 4 The hardware structure shown is described.

[0107] like Figure 4 As shown, the first electronic device 100 may include a processor 110, an external memory interface 120, an internal memory 121, a universal serial bus (USB) interface 130, a charging management module 140, a power management module 141, a battery 142, an antenna 1, an antenna 2, a mobile communication module 150, a wireless communication module 160, an audio module 170, a sensor module 180, a button 190, a motor 191, an indicator 192, a camera 193, a display screen 194, and a subscriber identification module (SIM) card interface 195, etc.

[0108] It is to be understood that the structure illustrated in the embodiment of the present application does not constitute a specific limitation on the first electronic device 100. In other embodiments of the present application, the first electronic device 100 may include more or fewer components than shown in the figure, or combine some components, or split some components, or arrange the components differently. The components shown in the figure may be implemented in hardware, software, or a combination of software and hardware.

[0109] The processor 110 may include one or more processing units, for example, the processor 110 may include an application processor (AP), a modem processor, a graphics processor (GPU), an image signal processor (ISP), a controller, a video codec, a digital signal processor (DSP), a baseband processor, and / or a neural-network processing unit (NPU), etc. Different processing units may be independent devices or integrated into one or more processors.

[0110] The controller can generate operation control signals according to the instruction operation code and timing signal to complete the control of instruction fetching and execution.

[0111] The processor 110 may also be provided with a memory for storing instructions and data. In some embodiments, the memory in the processor 110 is a cache memory. The memory may store instructions or data that the processor 110 has just used or cyclically used. If the processor 110 needs to use the instruction or data again, it may be directly called from the memory. This avoids repeated access, reduces the waiting time of the processor 110, and thus improves the efficiency of the system.

[0112] In some embodiments, the processor 110 may include one or more interfaces. The interface may include an inter-integrated circuit (I2C) interface, an inter-integrated circuit sound (I2S) interface, a pulse code modulation (PCM) interface, a universal asynchronous receiver / transmitter (UART) interface, a mobile industry processor interface (MIPI), a general-purpose input / output (GPIO) interface, a subscriber identity module (SIM) interface, and / or a universal serial bus (USB) interface, etc.

[0113] The MIPI interface can be used to connect the processor 110 with peripheral devices such as the display screen 194 and the camera 193. The MIPI interface includes a camera serial interface (CSI), a display serial interface (DSI), etc. In some embodiments, the processor 110 and the camera 193 communicate via the CSI interface to implement the shooting function of the first electronic device 100. The processor 110 and the display screen 194 communicate via the DSI interface to implement the display function of the first electronic device 100.

[0114] The USB interface 130 is an interface that complies with the USB standard specification, and specifically can be a Mini USB interface, a Micro USB interface, a USB Type C interface, etc. The USB interface 130 can be used to connect a charger to charge the first electronic device 100, and can also be used to transfer data between the first electronic device 100 and a peripheral device. It can also be used to connect headphones to play audio through the headphones. The interface can also be used to connect other electronic devices, such as AR devices, etc.

[0115] It is understandable that the interface connection relationship between the modules illustrated in the embodiment of the present application is only a schematic illustration and does not constitute a structural limitation on the first electronic device 100. In other embodiments of the present application, the first electronic device 100 may also adopt different interface connection methods in the above embodiments, or a combination of multiple interface connection methods.

[0116] The charging management module 140 is used to receive charging input from a charger. The charger may be a wireless charger or a wired charger. In some wired charging embodiments, the charging management module 140 may receive charging input from a wired charger through the USB interface 130. In some wireless charging embodiments, the charging management module 140 may receive wireless charging input through a wireless charging coil of the first electronic device 100. While the charging management module 140 is charging the battery 142, it may also power the electronic device through the power management module 141.

[0117] The power management module 141 is used to connect the battery 142, the charging management module 140 and the processor 110. The power management module 141 receives input from the battery 142 and / or the charging management module 140, and supplies power to the processor 110, the internal memory 121, the display screen 194, the camera 193, and the wireless communication module 160. The power management module 141 can also be used to monitor parameters such as battery capacity, battery cycle number, battery health status (leakage, impedance), etc. In some other embodiments, the power management module 141 can also be set in the processor 110. In other embodiments, the power management module 141 and the charging management module 140 can also be set in the same device.

[0118] The wireless communication function of the first electronic device 100 can be implemented through the antenna 1, the antenna 2, the mobile communication module 150, the wireless communication module 160, the modem processor and the baseband processor.

[0119] Antenna 1 and antenna 2 are used to transmit and receive electromagnetic wave signals. Each antenna in the first electronic device 100 can be used to cover a single or multiple communication frequency bands. Different antennas can also be reused to improve the utilization of the antennas. For example, antenna 1 can be reused as a diversity antenna for a wireless local area network. In some other embodiments, the antenna can be used in combination with a tuning switch.

[0120] The mobile communication module 150 can provide solutions for wireless communications including 2G / 3G / 4G / 5G applied to the first electronic device 100. The mobile communication module 150 may include at least one filter, a switch, a power amplifier, a low noise amplifier (LNA), etc. The mobile communication module 150 can receive electromagnetic waves from the antenna 1, and filter, amplify, and process the received electromagnetic waves, and transmit them to the modulation and demodulation processor for demodulation. The mobile communication module 150 can also amplify the signal modulated by the modulation and demodulation processor, and convert it into electromagnetic waves for radiation through the antenna 1. In some embodiments, at least some of the functional modules of the mobile communication module 150 can be set in the processor 110. In some embodiments, at least some of the functional modules of the mobile communication module 150 can be set in the same device as at least some of the modules of the processor 110.

[0121] The modem processor may include a modulator and a demodulator. The modulator is used to modulate the low-frequency baseband signal to be sent into a medium-high frequency signal. The demodulator is used to demodulate the received electromagnetic wave signal into a low-frequency baseband signal. The demodulator then transmits the demodulated low-frequency baseband signal to the baseband processor for processing. After the low-frequency baseband signal is processed by the baseband processor, it is passed to the application processor. The application processor outputs a sound signal through an audio device, or displays an image or video through a display screen 194. In some embodiments, the modem processor may be an independent device. In other embodiments, the modem processor may be independent of the processor 110 and be set in the same device as the mobile communication module 150 or other functional modules.

[0122] The wireless communication module 160 can provide wireless communication solutions including wireless local area networks (WLAN) (such as wireless fidelity (Wi-Fi) networks), bluetooth (BT), global navigation satellite system (GNSS), frequency modulation (FM), near field communication (NFC), infrared (IR), etc., applied on the first electronic device 100. The wireless communication module 160 can be one or more devices integrating at least one communication processing module. The wireless communication module 160 receives electromagnetic waves via the antenna 2, modulates the frequency of the electromagnetic wave signal and filters it, and sends the processed signal to the processor 110. The wireless communication module 160 can also receive the signal to be sent from the processor 110, modulate the frequency of it, amplify it, and convert it into electromagnetic waves for radiation through the antenna 2.

[0123] In some embodiments, the antenna 1 of the first electronic device 100 is coupled to the mobile communication module 150, and the antenna 2 is coupled to the wireless communication module 160, so that the first electronic device 100 can communicate with the network and other devices through wireless communication technology. The wireless communication technology may include global system for mobile communications (GSM), general packet radio service (GPRS), code division multiple access (CDMA), wideband code division multiple access (WCDMA), time-division code division multiple access (TD-SCDMA), long term evolution (LTE), BT, GNSS, WLAN, NFC, FM, and / or IR technology. The GNSS may include a global positioning system (GPS), a global navigation satellite system (GLONASS), a Beidou navigation satellite system (BDS), a quasi-zenith satellite system (QZSS) and / or a satellite based augmentation system (SBAS).

[0124] The first electronic device 100 implements a display function through a GPU, a display screen 194, and an application processor. The GPU is a microprocessor for image processing, which connects the display screen 194 and the application processor. The GPU is used to perform mathematical and geometric calculations for graphics rendering. The processor 110 may include one or more GPUs that execute program instructions to generate or change display information.

[0125] The display screen 194 is used to display images, videos, etc. The display screen 194 includes a display panel. The display panel can be manufactured using a liquid crystal display (LCD), such as an organic light-emitting diode (OLED), an active-matrix organic light-emitting diode or an active-matrix organic light-emitting diode (AMOLED), a flexible light-emitting diode (FLED), Mini-led, Micro-led, Micro-oled, quantum dot light emitting diodes (QLED), etc. In some embodiments, the first electronic device 100 may include 1 or N display screens 194, where N is a positive integer greater than 1.

[0126] The camera 193 is used to capture still images or videos. The object generates an optical image through the lens and projects it onto the photosensitive element. The photosensitive element can be a charge coupled device (CCD) or a complementary metal-oxide-semiconductor (CMOS) phototransistor. The photosensitive element converts the optical signal into an electrical signal, and then passes the electrical signal to the ISP to be converted into a digital image signal. The ISP outputs the digital image signal to the DSP for processing. The DSP converts the digital image signal into an image signal in a standard RGB, YUV or other format. In some embodiments, the first electronic device 100 may include 1 or N cameras 193, where N is a positive integer greater than 1.

[0127] The external memory interface 120 can be used to connect an external memory card, such as a Micro SD card, to expand the storage capacity of the first electronic device 100. The external memory card communicates with the processor 110 through the external memory interface 120 to implement a data storage function, such as storing music, video and other files in the external memory card.

[0128] The internal memory 121 can be used to store computer executable program codes, which include instructions. The internal memory 121 may include a program storage area and a data storage area. Among them, the program storage area may store an operating system, an application required for at least one function (such as a sound playback function, an image playback function, etc.), etc. The data storage area may store data created during the use of the first electronic device 100 (such as audio data, a phone book, etc.), etc. In addition, the internal memory 121 may include a high-speed random access memory, and may also include a non-volatile memory, such as at least one disk storage device, a flash memory device, a universal flash storage (UFS), etc. The processor 110 executes various functional applications and data processing of the first electronic device 100 by running instructions stored in the internal memory 121, and / or instructions stored in a memory provided in the processor.

[0129] The audio module 170 is used to convert digital audio information into analog audio signal output, and is also used to convert analog audio input into digital audio signal. The audio module 170 can also be used to encode and decode audio signals. In some embodiments, the audio module 170 can be arranged in the processor 110, or some functional modules of the audio module 170 can be arranged in the processor 110. The first electronic device 100 can use the audio module 170, such as music playing, recording, etc. The audio module 170 can include a speaker, a receiver, a microphone, a headphone interface, and an application processor to implement audio functions.

[0130] The sensor module 180 may include a pressure sensor, a gyro sensor, an air pressure sensor, a magnetic sensor, an acceleration sensor, a distance sensor, a proximity light sensor, a fingerprint sensor, a temperature sensor, a touch sensor, an ambient light sensor, a bone conduction sensor, and the like.

[0131] The key 190 includes a power key, a volume key, etc. The key 190 may be a mechanical key or a touch key. The first electronic device 100 may receive key input and generate key signal input related to user settings and function control of the first electronic device 100.

[0132] Motor 191 can generate vibration prompts. Motor 191 can be used for incoming call vibration prompts, and can also be used for touch vibration feedback. Motor 191 can also correspond to different vibration feedback effects for touch operations on different areas of display screen 194. Different application scenarios (for example: time reminder, receiving information, alarm clock, game, etc.) can also correspond to different vibration feedback effects.

[0133] The indicator 192 may be an indicator light, which may be used to indicate the charging status, power changes, messages, missed calls, notifications, etc.

[0134] The SIM card interface 195 is used to connect a SIM card. The SIM card can be connected to or disconnected from the first electronic device 100 by inserting or removing the SIM card interface 195. The first electronic device 100 can support 1 or N SIM card interfaces, where N is a positive integer greater than 1.

[0135] The following text takes the first electronic device 100 as a smart screen and the second electronic device 200 as a remote control handle as an example to explain in detail the three-dimensional interactive display method provided in the embodiment of the present application.

[0136] In some embodiments, the smart screen can obtain the positioning result of the remote controller, and determine the position, height and other information of the remote controller based on the positioning result. In some examples, the user holds the remote controller to interact with the smart screen, so the smart screen can determine the user's height, distance from the smart screen, arm length and other information based on the position and height of the remote controller, thereby providing the user with a 3D screen display that better meets the user's needs based on this user information.

[0137] In some examples, the 3D picture includes a static picture, a video picture, a displayed picture, a picture that can interact with the user, etc. For example, the 3D picture includes multiple types such as a static image or a frame image in a video picture.

[0138] For example, Figure 5 As shown, the smart screen is equipped with two remote controllers, which establish a communication connection with the smart screen through a wired connection or a wireless connection, and the same user operates the two remote controllers with both hands. Figure 5 As shown, the smart screen is equipped with a pointing and positioning transmitting module for sending positioning signals to the two remote controllers respectively. Accordingly, after receiving the positioning signal, the remote controller determines its own positioning result relative to the smart screen according to the positioning signal. Optionally, the positioning result includes information such as the direction of the remote controller and the distance relative to the smart screen. Afterwards, the remote controller sends the determined positioning result to the smart screen. Accordingly, as Figure 5 As shown, the smart screen receives the positioning results sent by the remote control handle through the wireless communication module.

[0139] It should be understood that the user generally does not move the position of the smart screen when using the smart screen, so the positioning result of the remote control handle relative to the smart screen obtained by the smart screen can be absolute positioning.

[0140] In another exemplary embodiment, the remote control handle sends a positioning signal to the smart screen in response to the user operation. Then, after receiving the positioning signal, the smart screen can generate a positioning result of the remote control handle according to the positioning signal. That is, the positioning result of the remote control handle can be determined by at least one device in the remote control handle or the smart screen.

[0141] In some examples, a positioning module is configured in the smart screen and / or remote control handle, and the positioning module can be used to send and / or receive positioning signals to generate positioning results.

[0142] In some embodiments, after the smart screen is started, through the interaction between the smart screen and the remote control handle, the smart screen can complete the initialization process of the remote control handle according to the positioning results obtained one or more times. During the initialization process, the smart screen can complete the positioning of the remote control handle, and can determine the physical information (such as height, arm length, etc.) of the user currently using the remote control handle. Thereby, in the subsequent use process, the smart screen can achieve more accurate interaction with the user. Optionally, the smart screen completes the initialization process according to a preset threshold number of positioning results. Alternatively, after the smart screen determines that the positioning of the remote control handle and the user's physical information have been obtained, it can determine that the initialization process is completed.

[0143] In some examples, during use, the smart screen determines that the deviation of the user's body information exceeds the preset threshold based on the obtained positioning results, and can determine that the user currently using the remote control handle has changed. Then, the smart screen can re-execute the initialization process by obtaining the positioning results. In this way, the usage data can be adaptively adjusted during use to meet the user's usage needs.

[0144] In some examples, the user will not change during the use of the smart screen. Therefore, after initialization, the smart screen no longer obtains the user's body information, but generates a corresponding 3D image display based on the positioning result of the remote control handle, thereby saving computing power.

[0145] In some examples, the smart screen can store information such as the user's physical information and the positioning information of the remote control handle. Therefore, when the smart screen responds to user operations and triggers the 3D interaction process (such as when the user starts a 3D game, etc.), the smart screen can directly realize the interaction between the 3D screen display and the user based on the historical storage information. In addition, the smart screen can also adaptively correct relevant information during the interaction process. Therefore, after the user turns on the machine, there is no need to calibrate the smart screen first, but the smart screen can be used directly for 3D display, thereby reducing the user's waiting. In addition, the smart screen can adaptively adjust relevant information during use to optimize the display effect and better meet the user's usage needs.

[0146] In some examples, the smart screen sends a positioning signal to the remote control handle according to a preset period, and the corresponding smart screen can obtain the positioning result of the remote control handle according to the preset period. Optionally, the smart screen can also obtain the positioning result according to the preset period by other means.

[0147] In this way, compared with VR head displays, which require users to manually perform initialization calibration after each startup and determine the safe zone before use, the three-dimensional interactive display method provided in the embodiment of the present application generally does not need to move the smart screen. Based on the storage and display characteristics of the smart screen, it is not necessary to force the user to perform the initialization calibration process after startup, which effectively simplifies user operations and reduces the difficulty of user use.

[0148] In some embodiments, the formation principle of human eye stereoscopic vision includes binocular stereo, monocular stereo (static), monocular motion stereo, and light and shadow effects. The human eye has the function of adjusting the line of sight (convergence) and adjusting the focal length of the lens, and these two functions correspond to binocular stereo formation and monocular stereo formation. The principle of binocular stereo formation is that when the user uses both eyes to watch, the extension lines of the binocular sight intersect at one point, which is the convergence point. When watching objects at different distances, the direction of the eye's sight is constantly adjusted, and the movement of the eyeball muscles and the coordinated information are synthesized into stereo information with spatial distance through the human brain. The principle of monocular stereo formation is that when the user uses one eye to watch, the formation of stereo is mainly related to the focus of the crystal. When watching objects at different distances, the focus of the eyeball crystal is also constantly adjusted until the image seen is clear (focused). When a person looks at a nearby object, the crystal will become thicker, and when looking at a distant object, it will become thinner. The movement of the eye muscles and the coordinated information are synthesized into stereo information with spatial distance through the human brain. The principle of monocular motion stereo is that when the information provided by the focus is insufficient (for example, when the distance is far and the focus does not change much), the human body will slightly adjust the position of the body or head, and use another perspective to provide auxiliary spatial sense information based on the relationship between the obstacles in front and behind the object. The principle of light and shadow effect is to use the light and shadow effects of an object in the real physical world to generate a three-dimensional display effect of the object in the image.

[0149] Based on the above-mentioned principle of the formation of human stereoscopic vision, the smart screen creates a stereoscopic display based on binocular parallax to achieve 3D picture display function. Among them, the distance between human eyes is about 6.5 cm. When people look at objects with both eyes, the left and right eyes see two slightly different images respectively, and the images formed on the retinas of the left and right eyes are also different. The comprehensive reaction of the cerebral cortex will produce a three-dimensional feeling. Therefore, the smart screen can let people see a three-dimensional picture by collecting / producing left and right eye images and inputting them to the left and right eyes respectively.

[0150] In some examples, the smart screen is integrated with VR software for generating 3D images. The VR software can generate 3D images based on the above principles and provide users with a 3D experience of the virtual world.

[0151] In some examples, after obtaining the positioning result of the remote controller, the smart screen sends the positioning result to the VR software. Then, in the process of generating 3D images, the VR software can combine the positioning result to generate a 3D image that meets the user's needs.

[0152] In some examples, the remote control handle can receive user operations and send the input data corresponding to the operations to the smart screen. Then, in the process of generating 3D images, the VR software in the smart screen can also combine the input data to generate corresponding 3D images, thereby enabling user interaction with the 3D images. Optionally, the operation includes, for example, the user's operation of buttons on the remote control handle, the user's hand movements (such as movement operations in different directions), etc.

[0153] Optionally, the smart screen displays corresponding virtual elements in the generated 3D picture according to the input data and positioning results sent by the remote control handle, and the virtual elements are used to indicate the user's operation of the remote control handle. Optionally, the virtual elements are displayed in the 3D picture as a cursor, a preset graphic, a track, etc. For example, when the user moves the remote control handle to the left, the smart screen displays the movement track of the remote control handle to the left on the display screen according to the change of the positioning result of the remote control handle.

[0154] Optionally, after the remote control handle generates input data in response to user operations, it can also provide feedback of operation effects to the user through vibration, key force feedback, etc., thereby making the user more immersed in the virtual world.

[0155] For example, Figure 5 As shown in the figure, VR software is integrated in the smart screen. After the wireless communication module receives the positioning results, input data and other information sent by the remote control handle, it sends the positioning results, input data and other information to the VR software. After that, the VR software generates a 3D picture based on the positioning results, input data and other information, and sends the 3D picture to the 3D display module for display.

[0156] In this way, after the smart screen generates a 3D image through VR software, it can be sent directly to the display module for display. There is no need for the image processing process of a VR headset to go through a deformation algorithm and a head tracking algorithm, thereby reducing the requirements for computing power and image processing delays, avoiding reduced display frame rate or abnormal freezes.

[0157] It should be understood that in addition to providing users with a 3D picture display experience, the smart screen can also provide users with a sound playback experience, allowing users to immerse themselves in the virtual world. This will not be elaborated below.

[0158] In some embodiments, after the smart screen generates a 3D image through VR software, it can display the 3D image. Then, the user can watch the 3D image displayed by the smart screen by wearing 3D glasses. In addition, during the viewing process, the user can interact with the 3D image by operating the remote control handle.

[0159] Optionally, users can watch the 3D images displayed on the smart screen by wearing glasses that use 3D technology, such as shutter glasses or polarized glasses.

[0160] Among them, shutter glass 3D technology belongs to active 3D technology. By using a pair of active LCD shutter glasses, the left and right eyes are switched on and off alternately, so that the left and right eyes of the user alternately see two images that merge into one in the brain, thereby providing the user with a 3D depth sense of one image. The principle of shutter 3D technology is based on the refresh time of the image frequency of the human eye. For example, the image refresh frequency needs to be greater than or equal to 120 Hz. Then, the left and right eyes quickly refresh the image at a frequency of 60Hz, so that the user will not feel the image jitter and maintain the same frame number as the 2D video. The user's two glasses see the rapid switching of different images, and an illusion is generated in the brain, so that the stereoscopic effect is seen.

[0161] Polarized (patterned retarder) 3D technology belongs to passive 3D technology, which is widely used in 3D cinemas, 3D TVs and other usage scenarios. For example, a polarizing film is attached to the display screen of a polarized 3D TV, so that the picture can be transmitted from different directions, and polarized glasses can separate the left and right eyes into vertical and horizontal pictures, which are overlapped in the user's brain to achieve a 3D effect.

[0162] For example, Figure 5 As shown, the smart screen generates 3D images through VR software and displays the 3D images through the display screen. Users can watch the 3D images displayed by the smart screen by wearing 3D glasses.

[0163] In this way, users can watch 3D images and interact with them by wearing 3D glasses. Compared with VR headsets, 3D glasses are much lighter and can provide the same immersive virtual world interactive experience as VR headsets. In addition, 3D glasses do not require external power supply, will not block the user's vision, and will not cause discomfort such as heating and stuffiness, thereby effectively improving user safety and comfort.

[0164] In other embodiments, the smart screen is equipped with a 3D display screen, and when the 3D images generated by the VR software are displayed on the 3D display screen, the user does not need to wear 3D glasses, and can directly watch the 3D images with naked eyes, thereby further reducing the user's operating difficulty and improving the user's experience.

[0165] Optionally, the current mainstream naked eye 3D technical means include slit liquid crystal grating, cylindrical lens, directional light source, and active backlight. Among them, the technical principle of the slit liquid crystal grating is to add a slit grating in front of the display screen, so that when the image seen by the left eye is displayed on the display screen during the image display process, the opaque stripes will block the right eye; similarly, when the image seen by the right eye is displayed on the display screen, the opaque stripes will block the left eye. In this way, by separating the visible images of the left eye and the right eye, the user can see the 3D picture. The technical principle of the cylindrical lens is to project the pixel points corresponding to the left and right eyes respectively into the left and right eyes through the refraction principle of the lens, so as to achieve image separation and enable the user to see the 3D picture. The technical principle of the directional light source is to accurately control the two sets of screens to project images to the left and right eyes respectively, so that the user can see the 3D picture. The technical principle of the active backlight is to use an optical microstructure to form a backlight module, and the light beam emitted by the backlight can adjust the direction under the control of electronic devices, so that the user can see the 3D picture.

[0166] In some embodiments, the smart screen establishes a communication connection with an external processing device and generates 3D images through VR software installed in the processing device, thereby reducing the computing power requirements for the smart screen and reducing the computing power consumption of the smart screen.

[0167] Optionally, the processing device is a terminal device such as a PC, a tablet, a mobile phone, etc. that has 3D image processing capabilities. A wired communication connection or a wireless communication connection is established between the smart screen and the processing device. For example, a wired communication connection is established between the smart screen and the processing device via USB, Ethernet, etc. For another example, a Wi-Fi wireless communication connection is established between the smart screen and the processing device.

[0168] For example, Figure 6As shown, a processing device is configured near the smart screen, and VR software is installed in the processing device. The smart screen sends a positioning signal to the remote control handle through the pointing positioning transmission module. After the remote control handle generates a positioning result based on the received positioning signal, the positioning result is fed back to the smart screen. After that, after the smart screen obtains the positioning result of the remote control handle through the wireless communication module, the positioning result is sent to the adaptation module, and the adaptation module is used to forward the obtained positioning result to the processing device. Optionally, the adaptation module is also used to forward the input data sent by the remote control handle received by the smart screen to the processing device, and the input data is the data generated by the user operating the remote control handle. Then, the processing device can generate a 3D picture based on the positioning results, input data and other information through VR software. Optionally, as described above, the positioning result includes information such as the distance between the remote control handle and the smart screen, the direction of the remote control handle, and the input data includes data generated by the user's button operation on the remote control handle, data generated by the user's hand movements while holding the remote control handle, and other data.

[0169] After that, the processing device can send the generated 3D picture to the smart screen. Correspondingly, after the smart screen receives the 3D picture sent by the processing device through the adaptation module, it displays the 3D picture through the 3D display module. Then, the user can watch the 3D picture displayed by the smart screen by wearing 3D glasses or naked eyes.

[0170] Optionally, the adaptation module can compress or decompress the data transmitted between the smart screen and the processing device to reduce the data transmission volume and improve data transmission efficiency.

[0171] In this way, by configuring external processing equipment with stronger computing power to obtain 3D images, the computing power requirements for smart screens are reduced. And the smart screen also directly displays the 3D images after obtaining them, without having to go through the deformation algorithm and head tracking algorithm processing, which has lower latency and resource requirements and further reduces the risk of lag.

[0172] Moreover, in the current solution, there is no need to force the user to perform an initialization calibration process after powering on the device, which effectively simplifies user operations and reduces user difficulty.

[0173] In addition, in the current solution, users can wear 3D glasses or watch 3D images with naked eyes, which can reduce the wearing weight, will not block the line of sight, 3D glasses do not require external power supply, and will not cause heat or stuffiness.

[0174] It should be understood that Figure 6 The 3D interactive display scene shown in the figure and the related implementation methods, principles and effects of the multiple 3D interactive display scenes described below can be referred to above. Figure 5The relevant contents described in the 3D interactive display scene shown are not repeated here. For example, the positioning result of the remote control handle can also be determined by the smart screen according to the positioning signal sent by the remote control handle.

[0175] In some embodiments, the emergence of multiplayer 3D games meets the needs of users for multiplayer game interaction. Limited by the single-person wearable usage characteristics of VR headsets, if users need to play multiplayer 3D games, then multiple users need to wear their own VR headsets and perform initialization calibration after powering on before they can play the game. In the three-dimensional interactive display method provided in the embodiments of the present application, by integrating the functions of the smart screen and the remote control handle, multiple users only need to wear different 3D glasses or directly use their naked eyes to watch the same 3D picture displayed by the smart screen, thereby realizing 3D picture sharing. Moreover, different users can control the virtual elements of the remote control handles corresponding to different users in the 3D picture by operating different remote control handles.

[0176] For example, Figure 7 As shown, user 1 and user 2 need to play 3D games, so user 1 can hold remote control handle 1, and user 2 can hold remote control handle 2. Afterwards, the smart screen can determine the positioning results of remote control handle 1 and remote control handle 2 by sending positioning signals to remote control handle 1 and remote control handle 2. In addition, the smart screen can receive input data sent by remote control handle 1 and remote control handle 2 respectively. Afterwards, the smart screen can generate a 3D picture through VR software based on the positioning results and input data and send it for display. In this way, user 1 and user 2 can watch the 3D picture displayed by the smart screen together by wearing 3D glasses or naked eyes, and can operate remote control handle 1 and remote control handle 2 respectively to interact with the smart screen.

[0177] It should be understood that the embodiment of the present application does not limit the number of remote control handles. For example, the user can configure a corresponding number of remote control handles according to needs.

[0178] In addition, the above method is also applicable to 3D display scenarios involving multiple people other than multiplayer 3D games.

[0179] In this way, multiple users can watch the same 3D picture by wearing 3D glasses or with naked eyes, thereby sharing the same picture, and can operate the same VR software through different remote control handles to achieve a multi-person interactive 3D virtual world experience.

[0180] Moreover, in multi-person 3D interactive scenarios, users do not need to purchase and wear multiple VR headsets, which reduces costs while providing the same user experience. And there are no disadvantages in using VR headsets.

[0181] In some embodiments, the 3D virtual world can provide users with a three-dimensional viewing experience. In the VR head display usage scenario, the VR head display can detect the user's head movement, generate a reverse motion picture through VR software, and virtualize a static perspective, thereby providing users with a viewing experience in which the picture moves with the user's head movement. However, the slight movement of the user's head can easily lead to unnecessary picture adjustments, which not only occupy system resources, but also cause the user to feel dizzy if the picture is not adjusted in time.

[0182] Based on this, the three-dimensional interactive display method provided in the embodiment of the present application meets the user's needs for adjusting the direction of the three-dimensional picture by adjusting the joystick or direction adjustment button (or touchpad) on the remote control handle and the change of field of view caused by the virtual person's head movement.

[0183] For example, Figure 8 As shown, a joystick and a joystick detection module for detecting the movement of the joystick are configured on the remote control handle. In some examples, the joystick can shake back and forth or left and right, or can rotate 360 ​​degrees and shake, etc. in response to user operations. The joystick detection module can convert the detected joystick shaking into input data. After that, the remote control handle can send the input data to the smart screen. After the smart screen receives the input data through the wireless communication module, it sends the input data to the VR software, and the VR software can convert the input data into the movement data of the user's head. Then, the VR software can generate a corresponding 3D picture of the field of view change caused by the user's head movement based on the movement data of the user's head, and send it for display. In this way, the user can see the 3D picture adjusted by the joystick through 3D glasses or naked eyes, and interact with the picture displayed on the smart screen.

[0184] For example, Fig. 9 As shown in the figure, the joystick detection module detects that the user shakes the joystick left or right, generates corresponding input data and sends it to the smart screen. The smart screen converts the input data into the pitch or tilt movement data of the user's head through VR software. For another example, Fig. 9 As shown in the figure, the joystick detection module detects that the user shakes the joystick forward or backward, generates corresponding input data and sends it to the smart screen. The smart screen converts the input data into rotational motion data of the user's head through VR software.

[0185] In this way, by converting the front and back or left and right inputs of the joystick through VR software, it can be equivalent to the 6DOF detection of the VR headset. When the user's head does not move, the screen perspective can be changed only by the joystick, thus avoiding the user's dizziness caused by low frame or frame loss.

[0186] In addition, the user's perspective is adjusted based on the joystick movement, and there is no need to generate reverse motion images, thereby reducing system resource usage, lowering processor requirements, and reducing the possibility of frame drops. In addition, the perspective of the image changes smoothly.

[0187] In some embodiments, some smart screens are equipped with a camera that can be used to collect user images. Then, in a 3D display scene, the smart screen can collect user images through the camera. Afterwards, the smart screen can determine the user positioning results based on the collected user images, such as parameters including the number of users, user positions, and user physical information (such as height, arm length, etc.). Based on the user positioning results and the positioning results of the remote control handle, the smart screen can more accurately determine the display position of the elements that need to be responded to in the 3D picture.

[0188] For example, the remote control handle is displayed as a virtual element in the 3D picture. In response to the user moving the remote control handle, the smart screen can change the display position of the virtual element in the 3D picture, thereby achieving interaction with the user.

[0189] As mentioned above, the smart screen can complete the initialization process of the smart screen based on one or more positioning results of the remote control handle. Then, the smart screen can effectively improve the initialization efficiency by combining the user positioning results.

[0190] For example, Fig.10 As shown, the smart screen is equipped with a calibration camera, which collects user images and determines the user positioning results. After that, the calibration camera sends the user positioning results to the VR software, and the VR software can combine the user positioning results, the remote control handle positioning results, and the input data of the remote control handle to generate a 3D picture and send it for display. In this way, users can watch 3D pictures by wearing 3D glasses or naked eyes, and the final 3D picture display can better meet the needs of users.

[0191] In this way, the smart screen is initialized through the camera, and the user does not need to participate in the initialization every time the device is turned on. While ensuring the accurate 3D display effect, the user's operation difficulty is reduced.

[0192] Fig.11 The following is a flow chart of a three-dimensional interactive display method provided in an embodiment of the present application. Fig.11 The specific order described below is a limitation, and it should be understood that in other embodiments, the order of some steps in the method can be interchanged according to actual needs, or some steps can be omitted or deleted. The method includes the following steps:

[0193] S1101. A first electronic device obtains a first positioning result of a second electronic device, where the first positioning result is determined by a wireless positioning signal.

[0194] The first positioning result includes at least one of the distance between the second electronic device and the first electronic device, the direction information of the second electronic device, and the height information of the second electronic device.

[0195] In some examples, the first electronic device is fixedly placed at a first position. In some examples, a spatial coordinate system is established based on the position of the first electronic device to obtain the three-dimensional coordinates and orientation information of the second electronic device relative to the first electronic device. For example, the three-dimensional coordinates and orientation information are the first positioning result of the second electronic device.

[0196] In some embodiments, the first positioning result is determined by the first electronic device based on the wireless positioning signal sent by the second electronic device. Alternatively, the first positioning result is determined by the second electronic device based on the wireless positioning signal sent by the first electronic device.

[0197] In some examples, a first positioning module is configured in the first electronic device, and a second positioning module is configured in the second electronic device. The first electronic device sends a first wireless positioning signal to the second electronic device through the first positioning module. Afterwards, the first electronic device receives a first positioning result sent by the second electronic device, and the first positioning result is determined by the second electronic device through the second positioning module according to the first wireless positioning signal. And / or, the first electronic device receives a second wireless positioning signal sent by the second electronic device through the second positioning module through the first positioning module. Afterwards, the first electronic device determines the first positioning result according to the second wireless positioning signal through the first positioning module.

[0198] Among them, the first positioning module or the second positioning module is any one of the following: UWB positioning module, electromagnetic positioning module, millimeter wave positioning module, optical tracking positioning module.

[0199] For example, Figure 5 As shown, the smart screen (such as the first electronic device) establishes a communication connection with the remote control handle (such as the second electronic device) through a wired connection or a wireless connection. The first electronic device is equipped with a pointing positioning transmission module (i.e., the first positioning module) for sending a wireless positioning signal to the second electronic device. Accordingly, after the second electronic device receives the wireless positioning signal, it determines its first positioning result relative to the first electronic device based on the wireless positioning signal. Accordingly, as Figure 5 As shown, the first electronic device receives the first positioning result sent by the second electronic device through the wireless communication module.

[0200] In this way, the first electronic device can achieve more accurate positioning of the second electronic device through the wireless positioning signal, so as to subsequently display a three-dimensional image that better meets the user's needs.

[0201] In some embodiments, before obtaining the first positioning result of the second electronic device, the first electronic device also obtains the location information of the first electronic device, and the location information is used for displaying a three-dimensional picture.

[0202] For example, when the first electronic device starts the VR software in the first electronic device for the first time, it obtains its own position information to initialize the VR software and adjust the viewing angle height of the user using the second electronic device in the three-dimensional picture. Optionally, the first electronic device can save the position information, so that the display of the three-dimensional picture can be adjusted according to the position information during the subsequent three-dimensional picture display process.

[0203] In this way, compared with VR head displays, which require users to manually perform initialization calibration after each startup and determine the safe zone before use, the three-dimensional interactive display method provided in the embodiment of the present application generally does not need to move the first electronic device. Based on the storage and display characteristics of the first electronic device, it is not necessary to force the user to perform the initialization calibration process after startup, which effectively simplifies user operations and reduces the difficulty of user use.

[0204] S1102: The first electronic device displays a three-dimensional picture, wherein the three-dimensional picture includes a first virtual element matching the first positioning result, and the three-dimensional picture is used for at least one user to watch or interact with outside the first electronic device.

[0205] In some embodiments, the first electronic device displays a first virtual element matching the first positioning result in the three-dimensional picture according to the first positioning result, thereby enabling the user to interact with the three-dimensional picture through the second electronic device. For example, the moving direction of the first virtual element is related to the pointing information of the second electronic device. For another example, the display position of the first virtual element in the three-dimensional picture is related to the moving position of the second electronic device operated by the user.

[0206] In some examples, the three-dimensional picture includes a static picture, a video picture, a displayed picture, a picture that can interact with a user, etc. For example, the three-dimensional picture includes multiple types such as a static image or a frame image in a video picture.

[0207] In some embodiments, the first virtual element is at least one of a display component, a controllable component, and a trajectory that matches the movement of the second electronic device.

[0208] For example, the first electronic device displays the first virtual element, that is, the first virtual element is a display component. For another example, the first electronic device controls the change of the first virtual element in the three-dimensional picture, such as the change of the display position, size, shape, etc., in response to the user's operation on the second electronic device, that is, the first virtual element is a controllable component. For another example, the first electronic device displays the movement track of the first virtual element in the process of moving the display position of the first virtual element in the three-dimensional picture in response to the user's operation on the second electronic device, that is, the first virtual element is a track that matches the movement of the second electronic device.

[0209] In some embodiments, the 3D image is used for viewing by the user outside the first electronic device, including: the 3D image is used for viewing by the user wearing 3D glasses or with naked eyes, and the 3D glasses include shutter glasses or polarized glasses.

[0210] For example, Figure 5 As shown, the first electronic device (such as a smart screen) generates a three-dimensional image through VR software and displays the three-dimensional image through a display screen. The user can watch the three-dimensional image displayed by the smart screen by wearing three-dimensional glasses.

[0211] In this way, users can watch 3D images and interact with them by wearing 3D glasses. Compared with VR headsets, 3D glasses are much lighter and can provide the same immersive virtual world interactive experience as VR headsets. In addition, 3D glasses do not require external power supply, which will not cause discomfort such as heating and stuffiness, thus effectively improving user comfort.

[0212] In addition, when users wear 3D glasses or watch 3D images with naked eyes, their vision will not be blocked and they can observe the real environment around them at the same time, thereby effectively improving their safety.

[0213] In some embodiments, the 3D image is provided for simultaneous viewing by multiple users.

[0214] In some embodiments, the first electronic device may also obtain a second positioning result of the third electronic device, and the three-dimensional picture also includes a second virtual element matching the second positioning result.

[0215] In some examples, the second electronic device and the third electronic device are operated by one or two users. For example, the second electronic device and the third electronic device are a pair of remote control handles, and the user can operate the two remote control handles with the left and right hands respectively. Then, the first virtual element and the second virtual element are the same virtual elements corresponding to the user. For another example, the second electronic device and the third electronic device are a pair of remote control handles, and two users operate the two remote control handles respectively. Then, the first virtual element and the second virtual element are different virtual elements corresponding to the two users respectively. It should be understood that the first electronic device can also be connected to more devices, so that more users can interact with the same three-dimensional picture displayed by the first electronic device by operating different devices. For example, the user can configure a corresponding number of remote control handles according to needs.

[0216] For example, Figure 7 As shown, user 1 and user 2 need to play 3D games, then user 1 can hold remote control handle 1 (such as the second electronic device), and user 2 can hold remote control handle 2 (such as the third electronic device). Afterwards, the smart screen (such as the first electronic device) can determine the positioning results of remote control handle 1 and remote control handle 2 by sending positioning signals to remote control handle 1 and remote control handle 2. In addition, the smart screen can receive input data sent by remote control handle 1 and remote control handle 2 respectively. Afterwards, the smart screen can generate a 3D picture through VR software based on the positioning results and input data and send it for display. In this way, user 1 and user 2 can watch the 3D picture displayed by the smart screen together by wearing 3D glasses or naked eyes, and can operate remote control handle 1 and remote control handle 2 to interact with the smart screen respectively.

[0217] In this way, multiple users can watch the same 3D picture by wearing 3D glasses or with naked eyes, thereby sharing the same picture, and can operate the same VR software through different devices to achieve a multi-person interactive 3D virtual world experience.

[0218] Moreover, in multi-person 3D interactive scenarios, users do not need to purchase and wear multiple VR headsets, which reduces costs while providing the same user experience. And there are no disadvantages in using VR headsets.

[0219] In some embodiments, the first electronic device receives input data sent by the second electronic device, wherein the input data includes at least one of key input data, input data corresponding to hand movements, and input data corresponding to joystick operations, and the input data is used to interact with the three-dimensional image.

[0220] In this way, the first electronic device updates the display of the three-dimensional picture according to the input data, thereby realizing the interaction between the user and the three-dimensional picture.

[0221] In some examples, the input data corresponding to the joystick operation is used to instruct to update the display content or display angle of the three-dimensional image.

[0222] For example, Figure 8 As shown, a joystick and a joystick detection module for detecting the movement of the joystick are configured on the remote control handle (such as the second electronic device). In some examples, the joystick can shake back and forth or left and right, or can rotate 360 ​​degrees in response to user operations. The joystick detection module can convert the detected joystick shaking into input data. After that, the remote control handle can send the input data to the smart screen (such as the first electronic device). After receiving the input data through the wireless communication module, the smart screen sends the input data to the VR software, and the VR software can convert the input data into the movement data of the user's head. Then, the VR software can generate a corresponding 3D picture of the field of view change caused by the user's head movement based on the movement data of the user's head, and send it for display. In this way, the user can see the 3D picture adjusted by the joystick through 3D glasses or naked eyes, and interact with the picture displayed by the smart screen.

[0223] For example, Fig. 9 As shown, the input data generated by the user shaking the joystick left or right can be converted into the pitch or tilt motion data of the user's head. The input data generated by the user shaking the joystick forward or backward can be converted into the rotation motion data of the user's head. In this way, the display content or display angle of the three-dimensional image can be updated through the joystick.

[0224] In this way, by converting the front and back or left and right inputs of the joystick through VR software, it can be equivalent to the 6DOF detection of the VR headset. When the user's head does not move, the screen perspective can be changed only by the joystick, thus avoiding the user's dizziness caused by low frame or frame loss.

[0225] In addition, the user's perspective is adjusted based on the joystick movement, and there is no need to generate reverse motion images, thereby reducing system resource usage, lowering processor requirements, and reducing the possibility of frame drops. In addition, the perspective of the image changes smoothly.

[0226] In some examples, the second electronic device is a device equipped with a display screen, such as a mobile phone, a tablet, etc. Then, the second electronic device can receive user operations by displaying virtual buttons and generate corresponding input data. For example, the second electronic device detects the user's operation of the up, down, left, and right controls displayed on the display screen, and generates corresponding input data, which can also be converted into the pitch or tilt motion data of the user's head. Thus, the second electronic device enables the user to interact with the three-dimensional picture displayed by the first electronic device through the display of virtual buttons. In some examples, the second electronic device determines the direction and position of the user moving the second electronic device through sensor data, and can generate corresponding positioning results.

[0227] In some other embodiments, the first electronic device obtains the third positioning result of the second electronic device. Afterwards, the first electronic device updates the display content or display angle of the three-dimensional image according to the third positioning result.

[0228] In this way, in response to the change of the positioning result of the second electronic device, the first electronic device can also update the display of the three-dimensional picture accordingly, thereby realizing the interaction between the user and the three-dimensional picture.

[0229] In some embodiments, the first electronic device collects user images through a camera, wherein the user images are used to determine at least one parameter of the number of users, the location of users, and the physical information of users.

[0230] For example, Fig.10 As shown, the smart screen (such as the first electronic device) is equipped with a calibration camera, through which the user image is collected and the user positioning result is determined. Afterwards, the calibration camera sends the user positioning result to the VR software, and the VR software can combine the user positioning result, the remote control handle positioning result, and the input data of the remote control handle to generate a three-dimensional picture and send it for display. In this way, the user can watch the three-dimensional picture by wearing three-dimensional glasses or naked eyes, and the display of the final three-dimensional picture can better meet the needs of the user.

[0231] In this way, the initialization of the first electronic device is completed through image acquisition of the first electronic device, and the user does not need to participate in the initialization every time after starting the device, thereby reducing the difficulty of user operation while ensuring the accurate effect of three-dimensional display.

[0232] In some embodiments, the first electronic device determines the three-dimensional image through the first software in the first electronic device according to the first positioning result, and the first software is the first VR software. That is, the first electronic device generates the three-dimensional image through the VR software configured inside it.

[0233] For example, Figure 5 As shown, the smart screen (such as the first electronic device) is integrated with VR software. After the wireless communication module receives the positioning results, input data and other information sent by the remote control handle (such as the second electronic device), the positioning results, input data and other information are sent to the VR software. Afterwards, the VR software generates a three-dimensional picture based on the positioning results, input data and other information, and sends the three-dimensional picture to the 3D display module for display.

[0234] In this way, after the first electronic device generates a three-dimensional image through VR software, it can be sent directly to the display module for display. There is no need for the image processing process of the VR headset to go through the deformation algorithm and the head tracking algorithm, thereby reducing the requirements for computing power and reducing the image processing delay, avoiding the display frame rate reduction or abnormal freezing.

[0235] In some other embodiments, the first electronic device sends the first positioning result to the fourth electronic device. Afterwards, the first electronic device receives a three-dimensional image fed back by the fourth electronic device, where the three-dimensional image is obtained by the fourth electronic device after processing the first positioning result through second software in the fourth electronic device, and the second software is a second VR software.

[0236] For example, Figure 6 As shown, a processing device (such as a fourth electronic device) is configured near the smart screen (such as the first electronic device), and VR software is installed in the processing device. The smart screen sends a positioning signal to the remote control handle by pointing to the positioning transmission module. After the remote control handle generates a positioning result based on the received positioning signal, it feeds back the positioning result to the smart screen. After that, after the smart screen obtains the positioning result of the remote control handle through the wireless communication module, the positioning result is sent to the adaptation module, and the adaptation module is used to forward the obtained positioning result to the processing device. Then, the processing device can generate a 3D picture based on the positioning result, input data and other information through the VR software. After that, the processing device can send the generated three-dimensional picture to the smart screen. Correspondingly, after the smart screen receives the three-dimensional picture sent by the processing device through the adaptation module, the three-dimensional picture is displayed through the 3D display module. Then, the user can watch the three-dimensional picture displayed by the smart screen by wearing three-dimensional glasses or naked eyes.

[0237] In some examples, the first VR software and the second VR software are the same VR software. Alternatively, the first VR software and the second VR software are different VR software adapted to different device types. Alternatively, the first VR software and the second VR software are different versions of VR software.

[0238] In this way, by configuring a fourth electronic device with stronger external computing power to obtain a 3D image, the computing power requirement for the first electronic device is reduced. The first electronic device also directly displays the 3D image after obtaining it, without having to go through the deformation algorithm and head tracking algorithm, which has lower requirements for delay and resources, and further reduces the risk of lag.

[0239] Optionally, the first electronic device can also execute the steps and functions performed by the smart screen in the above embodiments, and the second electronic device can also execute the steps and functions of the remote control handle in the above embodiments, thereby realizing the three-dimensional interactive display method provided in the above embodiments.

[0240] In some embodiments, the three-dimensional interactive display method provided in the embodiments of the present application can also be applied to a spliced ​​screen scenario, wherein the first electronic device is a spliced ​​screen device, the display screen of the first electronic device includes multiple spliced ​​screens, and the first electronic device displays a spliced ​​three-dimensional image according to the positioning result of the second electronic device.

[0241] In some examples, the user can interact with the spliced ​​three-dimensional picture displayed by the first electronic device through the second electronic device. For example, the user controls the first virtual element to move from one spliced ​​screen to another spliced ​​screen for display by moving the second electronic device. For another example, when the user controls the movement of the first virtual element through the second electronic device, the movement track covers multiple spliced ​​screens, and the first electronic device displays the spliced ​​movement track on multiple spliced ​​screens.

[0242] In some embodiments, the three-dimensional interactive display method provided in the embodiments of the present application can also be applied to multi-screen collaborative scenarios. Among them, the first electronic device can establish a communication connection with other electronic devices with display functions and realize display interaction. For example, the first electronic device is electronic device 1, and the electronic device that performs display interaction with the first electronic device is electronic device 2. Among them, electronic device 1 supports 3D display, and electronic device 2 supports 2D display or 3D display.

[0243] In some examples, while electronic device 1 is displaying a three-dimensional picture 1, in response to a user moving a second electronic device toward electronic device 2 to indicate an operation of displaying the first virtual element on electronic device 2, electronic device 1 may send display information of the three-dimensional picture 1 to electronic device 2.

[0244] When the electronic device 2 supports 3D display, the electronic device 2 can display the three-dimensional picture 2 according to the display information of the three-dimensional picture 1, and the three-dimensional picture 2 includes the first virtual element. Among them, the three-dimensional picture 2 is the same as or corresponds to the three-dimensional picture 1, so as to realize the continuous display of the display picture. In this way, the electronic device 1 realizes the three-dimensional picture interaction between different electronic devices according to the user operation, and enriches the user's interaction needs. For example, during the movement of the first virtual element, the electronic device 2 is being operated by another user, so that the three-dimensional picture 2 displayed by the electronic device 2 may include the first virtual element and another virtual element corresponding to the other user.

[0245] When the electronic device 2 supports 2D display, the electronic device 2 can display a two-dimensional picture according to the display information of the three-dimensional picture 1, and the two-dimensional picture includes a two-dimensional virtual element corresponding to the first virtual element. The two-dimensional picture corresponds to the three-dimensional picture 1. In this way, the electronic device 1 realizes the interaction of display pictures between different electronic devices according to the user operation, and further enriches the user's interaction needs.

[0246] In other examples, electronic device 2 displays a two-dimensional screen, which includes a two-dimensional virtual element corresponding to user 1. In response to the user operation, electronic device 2 determines to move the two-dimensional virtual element to electronic device 1 for display, and electronic device 2 may send display information of the two-dimensional screen to electronic device 1. Correspondingly, after receiving the display information of the two-dimensional screen, electronic device 1 may display a corresponding three-dimensional screen according to the display information, and the three-dimensional screen includes a three-dimensional virtual element corresponding to the two-dimensional virtual element.

[0247] In this way, when the electronic device supports 2D display and / or 3D display, different electronic devices can respond to user operations to achieve interaction of display images, thereby enriching the interaction scenarios and meeting the user's usage needs.

[0248] It should be understood that a multi-screen collaboration scenario may include two or more electronic devices, which may support 2D display or 3D display, or 4D or higher display. In response to the user's operation on the second electronic device (such as a remote control handle, etc.), different electronic devices can display corresponding display screens through the interaction of display information to meet the user's interaction needs. This embodiment of the application will not be exemplified one by one.

[0249] Combination of the above Figure 5-Figure 11 The three-dimensional interactive display method provided by the embodiment of the present application is described in detail. Fig.12 The first electronic device provided in an embodiment of the present application is described in detail.

[0250] In one possible design, Fig.12 This is a schematic diagram of the structure of a first electronic device provided in an embodiment of the present application. Fig.12 As shown, the first electronic device 1200 may include: a transceiver unit 1201, a processing unit 1202, and a display unit 1203. The first electronic device 1200 may be used to implement the functions of the first electronic device (such as a smart screen) involved in the above method embodiment.

[0251] Optionally, the transceiver unit 1201 is used to support the first electronic device 1200 to execute Fig.11 S1101 in.

[0252] Optionally, the processing unit 1202 is used to support the first electronic device 1200 to execute Fig.11 S1101 in.

[0253] Optionally, the display unit 1203 is used to support the first electronic device 1200 to execute Fig.11 S1102 in.

[0254] Among them, the transceiver unit may include a receiving unit and a sending unit, and may be implemented by a transceiver or a transceiver-related circuit component, and may be a transceiver or a transceiver module. The operations and / or functions of each unit in the first electronic device 1200 are respectively to implement the corresponding process of the three-dimensional interactive display method described in the above method embodiment. All relevant contents of each step involved in the above method embodiment can be referred to the functional description of the corresponding functional unit, and for the sake of brevity, they will not be repeated here.

[0255] Optionally, Fig.12 The first electronic device 1200 shown may also include a storage unit ( Fig.12 (not shown), the storage unit stores a program or instruction. When the transceiver unit 1201, the processing unit 1202 and the display unit 1203 execute the program or instruction, Fig.12 The first electronic device 1200 shown can execute the three-dimensional interactive display method described in the above method embodiment.

[0256] Fig.12 The technical effects of the first electronic device 1200 shown can refer to the technical effects of the three-dimensional interactive display method described in the above method embodiment, and will not be repeated here.

[0257] In addition to being in the form of the first electronic device 1200, the technical solution provided in the present application may also be a functional unit or chip in the first electronic device, or a device used in conjunction with the first electronic device.

[0258] An embodiment of the present application also provides a chip system, including: a processor, the processor is coupled to a memory, the memory is used to store programs or instructions, when the program or instructions are executed by the processor, the chip system implements the method in any of the above method embodiments.

[0259] Optionally, the processor in the chip system may be one or more. The processor may be implemented by hardware or by software. When implemented by hardware, the processor may be a logic circuit, an integrated circuit, etc. When implemented by software, the processor may be a general-purpose processor implemented by reading software code stored in a memory.

[0260] Optionally, the memory in the chip system may also be one or more. The memory may be integrated with the processor or may be separately arranged with the processor, which is not limited in the embodiments of the present application. Exemplarily, the memory may be a non-transient processor, such as a read-only memory ROM, which may be integrated with the processor on the same chip or may be arranged on different chips respectively. The embodiments of the present application do not specifically limit the type of memory and the arrangement of the memory and the processor.

[0261] Exemplarily, the chip system can be a field programmable gate array (FPGA), an application specific integrated circuit (ASIC), a system on chip (SoC), a central processor unit (CPU), a network processor (NP), a digital signal processor (DSP), a microcontroller unit (MCU), a programmable logic device (PLD) or other integrated chips.

[0262] It should be understood that each step in the above method embodiment can be completed by an integrated logic circuit of hardware in a processor or by instructions in the form of software. The method steps disclosed in the embodiments of the present application can be directly embodied as being executed by a hardware processor, or by a combination of hardware and software modules in a processor.

[0263] An embodiment of the present application also provides a computer-readable storage medium, in which a computer program is stored. When the computer program is run on a computer, the computer executes the above-mentioned related steps to implement the three-dimensional interactive display method in the above-mentioned embodiment.

[0264] The embodiment of the present application also provides a computer program product. When the computer program product is run on a computer, the computer is caused to execute the above-mentioned related steps to implement the three-dimensional interactive display method in the above-mentioned embodiment.

[0265] In addition, an embodiment of the present application further provides a device. The device may be a component or a module, and the device may include one or more processors and a memory connected to each other. The memory is used to store a computer program. When the computer program is executed by one or more processors, the device performs the three-dimensional interactive display method in the above-mentioned method embodiments.

[0266] Among them, the device, computer-readable storage medium, computer program product or chip provided in the embodiments of the present application are used to execute the corresponding methods provided above. Therefore, the beneficial effects that can be achieved can refer to the beneficial effects in the corresponding methods provided above, and will not be repeated here.

[0267] The steps of the method or algorithm described in conjunction with the disclosure of the embodiments of the present application can be implemented in hardware or by executing software instructions by a processor. The software instructions can be composed of corresponding software modules, and the software modules can be stored in random access memory (RAM), flash memory, read only memory (ROM), erasable programmable read-only memory (EPROM), electrically erasable programmable read-only memory (EEPROM), registers, hard disks, mobile hard disks, CD-ROMs or any other form of storage medium known in the art. An exemplary storage medium is coupled to a processor so that the processor can read information from the storage medium and can write information to the storage medium. Of course, the storage medium can also be a component of the processor. The processor and the storage medium can be located in an application specific integrated circuit (ASIC).

[0268] Through the description of the above implementation methods, those skilled in the art can clearly understand that for the convenience and simplicity of description, only the division of the above functional modules is used as an example. In practical applications, the above functions can be assigned to different functional modules as needed; that is, the internal structure of the device is divided into different functional modules to complete all or part of the functions described above. The specific working process of the system, device and unit described above can refer to the corresponding process in the aforementioned method embodiment, and will not be repeated here.

[0269] In the several embodiments provided in the present application, it should be understood that the disclosed method can be implemented in other ways. The device embodiments described above are merely schematic. For example, the division of the modules or units is only a logical function division, and there may be other division methods in actual implementation; for example, multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. In addition, the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, indirect coupling or communication connection of modules or units, which can be electrical, mechanical or other forms.

[0270] In addition, each functional unit in each embodiment of the present application may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit. The above-mentioned integrated unit may be implemented in the form of hardware or in the form of software functional units.

[0271] Computer-readable storage media include, but are not limited to, any of the following: a USB flash drive, a mobile hard disk, a read-only memory (ROM), a random access memory (RAM), a magnetic disk, or an optical disk, and other media that can store program codes.

[0272] The above is only a specific implementation of the present application, but the protection scope of the present application is not limited thereto. Any changes or substitutions within the technical scope disclosed in the present application should be included in the protection scope of the present application. Therefore, the protection scope of the present application should be based on the protection scope of the claims.

Claims

1. A three-dimensional interactive display method, characterized in that: Applied to a first electronic device, the first electronic device is fixedly placed at a first position, the method includes: Acquire a first positioning result of the second electronic device, where the first positioning result is determined by a wireless positioning signal; A three-dimensional picture is displayed, wherein the three-dimensional picture includes a first virtual element matching the first positioning result, and the three-dimensional picture is used for at least one user to watch or interact outside the first electronic device.

2. The method according to claim 1, characterized in that The three-dimensional image is used for multiple users to watch simultaneously.

3. The method according to claim 1 or 2, characterized in that: The method further comprises: A second positioning result of the third electronic device is obtained, and the three-dimensional picture also includes a second virtual element matching the second positioning result.

4. The method according to claim 3, characterized in that The second electronic device and the third electronic device are operated by one or two users.

5. The method according to any one of claims 1 to 4, characterized in that: The first virtual element is at least one of a display component, a controllable component, and a trajectory matching the movement of the second electronic device.

6. The method according to any one of claims 1 to 5, characterized in that: The method further comprises: Receive input data sent by the second electronic device, where the input data includes at least one of key input data, input data corresponding to hand movements, and input data corresponding to joystick operations, and the input data is used to interact with the three-dimensional image.

7. The method according to claim 6, characterized in that The input data corresponding to the joystick operation is used to instruct to update the display content or display angle of the three-dimensional image.

8. The method according to any one of claims 1 to 5, characterized in that: The method further comprises: Obtaining a third positioning result of the second electronic device; According to the third positioning result, the display content or the display angle of the three-dimensional image is updated.

9. The method according to any one of claims 1 to 8, characterized in that: Before displaying the three-dimensional picture, the method further includes: According to the first positioning result, the three-dimensional image is determined by first software in the first electronic device, and the first software is first virtual reality VR software.

10. The method according to any one of claims 1 to 8, characterized in that: Before displaying the three-dimensional picture, the method further includes: Sending the first positioning result to a fourth electronic device; The three-dimensional image fed back by the fourth electronic device is received, where the three-dimensional image is obtained after the fourth electronic device processes the first positioning result through second software in the fourth electronic device, and the second software is second VR software.

11. The method according to any one of claims 1 to 10, characterized in that: After obtaining the first positioning result of the second electronic device, the method further includes: The user image is collected by a camera, and the user image is used to determine at least one parameter of the number of users, the location of users, and the physical information of users.

12. The method according to any one of claims 1 to 11, characterized in that: Before acquiring the first positioning result of the second electronic device, the method further includes: The position information of the first electronic device is acquired, where the position information is used for displaying the three-dimensional image.

13. The method according to any one of claims 1 to 12, characterized in that: The first positioning result includes at least one of the distance between the second electronic device and the first electronic device, the direction information of the second electronic device, and the height information of the second electronic device.

14. The method according to any one of claims 1 to 13, characterized in that: The first electronic device is provided with a first positioning module, the second electronic device is provided with a second positioning module, and the obtaining of a first positioning result of the second electronic device includes: Sending a first wireless positioning signal to the second electronic device through the first positioning module; receiving a first positioning result sent by the second electronic device, where the first positioning result is determined by the second electronic device through the second positioning module according to the first wireless positioning signal; and / or, Receiving, through the first positioning module, a second wireless positioning signal sent by the second electronic device through the second positioning module; The first positioning result is determined by the first positioning module according to the second wireless positioning signal.

15. The method according to claim 14, characterized in that The first positioning module or the second positioning module is any one of the following: an ultra-wideband technology UWB positioning module, an electromagnetic positioning module, a millimeter wave positioning module, and an optical tracking positioning module.

16. The method according to any one of claims 1 to 15, characterized in that: The three-dimensional image is used for at least one user to watch outside the first electronic device, including: the three-dimensional image is used for at least one user to watch wearing three-dimensional glasses or watching with naked eyes, and the three-dimensional glasses include shutter glasses or polarized glasses.

17. An electronic device, characterized in that: include: A processor, a memory and a display screen, wherein the memory and the display screen are coupled to the processor, the memory is used to store computer program code, and the computer program code includes computer instructions. When the processor reads the computer instructions from the memory, the electronic device executes the method as described in any one of claims 1 to 16.

18. A computer-readable storage medium, characterized in that: The computer-readable storage medium comprises a computer program, and when the computer program is executed on an electronic device, the electronic device is enabled to execute the method according to any one of claims 1 to 16.

19. A computer program product, characterized in that When the computer program product is executed on a computer, the computer is caused to execute the method according to any one of claims 1 to 16.

Images