Three-dimensional interactive display method and electronic device

Abstract

A three-dimensional interactive display method and an electronic device, which relate to the technical field of terminals, and can provide three-dimensional display and interaction experience for a user by means of combining the three-dimensional display capability of a first electronic device and by means of interaction with a second electronic device, wherein the first electronic device is fixedly placed at a first position. The method comprises: a first electronic device acquiring a first positioning result of a second electronic device, wherein the first positioning result is determined by means of a wireless positioning signal; and then, the first electronic device displaying a three-dimensional picture, wherein the three-dimensional picture comprises a first virtual element matching the first positioning result, and the three-dimensional picture is used for at least one user to view or interact outside the first electronic device.

Description

Three-dimensional interactive display method and electronic device

[0001] This application claims priority to the Chinese patent application filed with the State Intellectual Property Office on August 30, 2023, with application number 202311112575.8 and application name “Three-dimensional interactive display method and electronic device”, the entire contents of which are incorporated by reference into this application. Technical Field

[0002] The present application relates to the field of terminal technology, and in particular to a three-dimensional interactive display method and electronic device. Background Art

[0003] With the development of terminal technology, some electronic devices can provide users with a three-dimensional (3D) display experience. For example, smart screens support 3D display, and users can use smart screens to watch 3D movies. However, smart screens cannot provide users with a 3D interactive experience. For example, VR games developed based on virtual reality (VR) technology allow users to view the 3D game interface by wearing a VR headset, and interact with 3D game elements through a controller that works with the VR headset. However, VR headsets are heavy and can easily cause user fatigue, and users cannot observe the real environment around them after wearing a 3D headset, which can easily cause danger.

[0004] Summary of the Invention

[0005] To address the aforementioned technical issues, this application provides a 3D interactive display method and electronic device. The technical solution provided by this application combines the 3D display capabilities of a first electronic device with interaction with a second electronic device to provide users with a 3D display and interactive experience. Furthermore, users can view 3D images outside the first electronic device, reducing the burden of wearing the device.

[0006] In order to achieve the above technical objectives, this application provides the following technical solutions:

[0007] In a first aspect, a three-dimensional interactive display method is provided, applied to a first electronic device. The first electronic device is fixedly placed at a first location. The method includes obtaining a first positioning result of a second electronic device, where the first positioning result is determined using a wireless positioning signal. A three-dimensional image is displayed, the three-dimensional image including a first virtual element that matches the first positioning result. The three-dimensional image is viewed or interacted with by at least one user outside the first electronic device.

[0008] In some examples, a first electronic device is fixedly placed at a first location, a spatial coordinate system is established based on the location of the first electronic device, and three-dimensional coordinates and orientation information of a second electronic device relative to the first electronic device are obtained. For example, a first positioning result of the second electronic device includes the three-dimensional coordinates and orientation information.

[0009] In this way, the first electronic device includes a first virtual element that matches the first positioning result in the displayed three-dimensional image based on the first positioning result of the second electronic device, thereby enabling user interaction with the three-dimensional image displayed by the first electronic device. For example, the movement direction of the first virtual element is related to the directional information of the second electronic device. For another example, the display position of the first virtual element in the three-dimensional image is related to the movement position of the second electronic device operated by the user.

[0010] Furthermore, the user can view the 3D image outside the first electronic device, thereby preventing the user's vision from being completely blocked. The user can view the real-world environment while viewing the 3D image, thereby improving user safety.

[0011] In some examples, the 3D screen includes an image screen, a video screen, an application display screen, a screen that can interact with the user, etc. For example, the 3D screen includes multiple types such as a static image or a frame image in a video screen.

[0012] According to the first aspect, or any implementation of the first aspect above, the method further includes: obtaining a second positioning result of the third electronic device, and the three-dimensional picture also includes a second virtual element matching the second positioning result.

[0013] According to the first aspect, a three-dimensional picture is provided for simultaneous viewing by multiple users.

[0014] According to the first aspect, or any implementation of the first aspect, the second electronic device and the third electronic device are operated by one or two users.

[0015] In some examples, the second electronic device and the third electronic device are operated by one or two users. For example, the second electronic device and the third electronic device are a pair of remote control handles, and the user can operate the two remote control handles with his left and right hands respectively. Then, the first virtual element and the second virtual element are the same virtual elements corresponding to the user. For another example, the second electronic device and the third electronic device are a pair of remote control handles, and two users operate the two remote control handles respectively. Then, the first virtual element and the second virtual element are different virtual elements corresponding to the two users respectively. It should be understood that the first electronic device can also be connected to more devices, so that more users can interact with the same three-dimensional picture displayed by the first electronic device by operating different devices. For example, the user can configure a corresponding number of remote control handles according to needs.

[0016] In this way, multiple users can watch the same three-dimensional picture by wearing three-dimensional glasses or with naked eyes, sharing the same picture, and can operate the same VR software through different devices to achieve a multi-person interactive three-dimensional virtual world experience.

[0017] Furthermore, in multi-person 3D interactive scenarios, users do not need to purchase and wear multiple VR headsets, reducing costs while still providing the same user experience. This eliminates the common shortcomings of VR headsets.

[0018] According to the first aspect, or any implementation of the first aspect above, the first virtual element is at least one of a display component, a controllable component, and a trajectory matching the movement of the second electronic device.

[0019] For example, the first electronic device displays the first virtual element, i.e., the first virtual element is a display component. For another example, the first electronic device controls changes in the first virtual element in a three-dimensional image, such as changes in display position, size, shape, etc., in response to a user's operation on a second electronic device. In this case, the first virtual element is a controllable component. For another example, the first electronic device displays a movement trajectory of the first virtual element in the process of moving the display position of the first virtual element in a three-dimensional image in response to a user's operation on a second electronic device. In this case, the first virtual element is a trajectory that matches the movement of the second electronic device.

[0020] According to the first aspect, or any implementation of the first aspect above, the method further includes: receiving input data sent by a second electronic device, the input data including at least one of key input data, input data corresponding to a hand movement, and input data corresponding to a joystick operation. The input data is used to interact with the three-dimensional image.

[0021] According to the first aspect, or any implementation of the first aspect above, the input data corresponding to the joystick operation is used to instruct to update the display content or display angle of the three-dimensional image.

[0022] By converting the forward and backward or left and right inputs of the joystick through VR software, it can be equivalent to 6DOF detection for the VR headset. This allows the user to change the viewing angle using only the joystick without moving their head, thus avoiding the feeling of sickness caused by low or dropped frames.

[0023] Furthermore, adjusting the user's perspective based on joystick movement eliminates the need to generate inverse motion images, thereby reducing system resource usage, placing lower demands on the processor, and minimizing the likelihood of frame drops. Furthermore, the perspective changes are smoother.

[0024] In some examples, the second electronic device is a device equipped with a display screen, such as a mobile phone, tablet, etc. Then, the second electronic device can receive user operations by displaying virtual buttons and generate corresponding input data. For example, the second electronic device detects the user's operation of the up, down, left, and right controls displayed on the display screen and generates corresponding input data, which can also be converted into pitch or tilt motion data of the user's head. Thus, the second electronic device enables the user to interact with the three-dimensional picture displayed by the first electronic device through the display of virtual buttons. In some examples, the second electronic device determines the direction and position of the user moving the second electronic device through sensor data and can generate corresponding positioning results.

[0025] According to the first aspect, or any implementation of the first aspect above, the method further includes: obtaining a third positioning result of the second electronic device, and updating the display content or display angle of the three-dimensional image according to the third positioning result.

[0026] In this way, in response to changes in the positioning result of the second electronic device, the first electronic device can also update the display of the three-dimensional image accordingly, thereby enabling interaction between the user and the three-dimensional image.

[0027] According to the first aspect, or any implementation of the first aspect above, before displaying the three-dimensional picture, the method also includes: determining the three-dimensional picture through first software in the first electronic device based on the first positioning result, where the first software is first virtual reality VR software.

[0028] In this way, after the first electronic device generates a three-dimensional image through VR software, it can be sent directly to the display module for display. There is no need for the image processing process of the VR headset to go through the deformation algorithm and head tracking algorithm, thereby reducing the requirements for computing power and reducing the image processing delay, avoiding the display frame rate reduction or abnormal freezing.

[0029] According to the first aspect, or any implementation of the first aspect above, before displaying the three-dimensional image, the method further includes: sending the first positioning result to a fourth electronic device, and receiving a three-dimensional image fed back by the fourth electronic device, where the three-dimensional image is obtained by the fourth electronic device after processing the first positioning result using second software in the fourth electronic device, where the second software is second VR software.

[0030] By configuring a fourth electronic device with greater computing power to capture 3D images, the computing power requirements of the first electronic device are reduced. Furthermore, the first electronic device can directly display the captured 3D images without having to go through the deformation and head tracking algorithms, thus reducing latency and resource requirements and further minimizing the risk of lag.

[0031] According to the first aspect, or any implementation of the first aspect above, after obtaining the first positioning result of the second electronic device, the method also includes: collecting user images through a camera, and the user images are used to determine at least one parameter of the number of users, user locations, and user physical information.

[0032] In this way, the initialization of the first electronic device is completed through image acquisition of the first electronic device, and the user does not need to participate in the initialization every time after starting the device. While ensuring the accurate effect of the three-dimensional display, the user's operation difficulty is reduced.

[0033] According to the first aspect, or any implementation of the first aspect above, before obtaining the first positioning result of the second electronic device, the method further includes: obtaining location information of the first electronic device, where the location information is used for displaying a three-dimensional image.

[0034] For example, when the first electronic device first launches VR software on the first electronic device, it obtains its own position information to initialize the VR software and adjust the viewing angle height of the user using the second electronic device in the three-dimensional image. Optionally, the first electronic device can save this position information so that the display of the three-dimensional image can be adjusted based on this position information during the subsequent three-dimensional image display process.

[0035] In this way, compared to VR headsets, which require users to manually perform initialization calibration after each power-on and determine the safe zone before use, the three-dimensional interactive display method provided in the embodiments of the present application generally does not require the first electronic device to move. Based on the storage and display characteristics of the smart screen, there is no need to force the user to perform the initialization calibration process after powering on, effectively simplifying user operations and reducing user difficulty.

[0036] According to the first aspect, or any implementation of the first aspect above, the first positioning result includes at least one of the distance between the second electronic device and the first electronic device, the orientation information of the second electronic device, and the height information of the second electronic device.

[0037] According to the first aspect, or any implementation of the first aspect above, a first positioning module is configured in a first electronic device, and a second positioning module is configured in a second electronic device. Obtaining a first positioning result for the second electronic device includes: sending a first wireless positioning signal to the second electronic device via the first positioning module. Receiving the first positioning result sent by the second electronic device, where the first positioning result is determined by the second electronic device via the second positioning module based on the first wireless positioning signal. And / or, receiving, via the first positioning module, a second wireless positioning signal sent by the second electronic device via the second positioning module. Determining the first positioning result via the first positioning module based on the second wireless positioning signal.

[0038] In this way, the first electronic device can achieve more accurate positioning of the second electronic device through the wireless positioning signal, so as to subsequently display a three-dimensional image that better meets the user's needs.

[0039] According to the first aspect, or any implementation of the first aspect above, the first positioning module or the second positioning module is any one of the following: an ultra-wideband technology UWB positioning module, an electromagnetic positioning module, a millimeter wave positioning module, or an optical tracking positioning module.

[0040] According to the first aspect, or any implementation of the first aspect above, the three-dimensional picture is used for at least one user to watch outside the first electronic device, including: the three-dimensional picture is used for at least one user to watch while wearing three-dimensional glasses or with naked eyes, and the three-dimensional glasses include shutter glasses or polarized glasses.

[0041] In this way, users can view and interact with 3D images simply by wearing 3D glasses. Compared to VR headsets, 3D glasses are significantly lighter and offer the same immersive virtual world experience. Furthermore, 3D glasses do not require an external power supply, which would otherwise cause discomfort such as heat and stuffiness, thus effectively improving user comfort.

[0042] In addition, when users wear 3D glasses or watch 3D images with naked eyes, their vision will not be blocked. Users can observe the real environment around them at the same time, thereby effectively improving user safety.

[0043] In some embodiments, the 3D interactive display method provided by the embodiments of the present application can also be applied to a spliced ​​screen scenario, where the first electronic device is a spliced ​​screen device, the display screen of the first electronic device includes multiple spliced ​​screens, and the first electronic device displays a spliced ​​3D image based on the positioning result of the second electronic device.

[0044] In some examples, a user can interact with the spliced ​​3D image displayed by the first electronic device through a second electronic device. For example, the user can control the movement of a first virtual element from one spliced ​​screen to another spliced ​​screen by moving the second electronic device. In another example, when the user controls the movement of the first virtual element through the second electronic device, the movement trajectory covers multiple spliced ​​screens, and the first electronic device displays the spliced ​​movement trajectory on the multiple spliced ​​screens.

[0045] In some embodiments, the three-dimensional interactive display method provided in the embodiments of the present application can also be applied to multi-screen collaborative scenarios. Among them, the first electronic device can establish a communication connection with other electronic devices with display functions and realize display interaction. For example, the first electronic device is electronic device 1, and the electronic device that performs display interaction with the first electronic device is electronic device 2. Among them, electronic device 1 supports 3D display, and electronic device 2 supports 2D display or 3D display.

[0046] In some examples, while electronic device 1 is displaying a three-dimensional picture 1, in response to a user moving a second electronic device toward a first virtual element of electronic device 2, indicating an operation to display the first virtual element on electronic device 2, electronic device 1 may send display information of the three-dimensional picture 1 to electronic device 2.

[0047] If electronic device 2 supports 3D display, electronic device 2 can display 3D image 2 based on the display information of 3D image 1, and 3D image 2 includes a first virtual element. 3D image 2 is the same as or corresponds to 3D image 1, thereby achieving continuous display of the displayed images. In this way, electronic device 1 enables 3D image interaction between different electronic devices based on user operations, further enriching user interaction needs. For example, while the first virtual element is moving, electronic device 2 is being operated by another user. In this way, the 3D image 2 displayed by electronic device 2 may include the first virtual element and another virtual element corresponding to the other user.

[0048] If electronic device 2 supports 2D display, it can display a 2D image based on the display information of 3D image 1. This 2D image includes a 2D virtual element corresponding to the first virtual element. The 2D image corresponds to 3D image 1. In this way, electronic device 1 enables interactive display between different electronic devices based on user operations, further enriching user interaction needs.

[0049] In other examples, electronic device 2 displays a two-dimensional image that includes a two-dimensional virtual element corresponding to user 1. In response to a user operation, electronic device 2 determines to move the two-dimensional virtual element to electronic device 1 for display, and electronic device 2 may send display information of the two-dimensional image to electronic device 1. Accordingly, after receiving the display information of the two-dimensional image, electronic device 1 may display a corresponding three-dimensional image based on the display information, which includes the three-dimensional virtual element corresponding to the two-dimensional virtual element.

[0050] In this way, when electronic devices support 2D display and / or 3D display, different electronic devices can respond to user operations and realize interaction of display screens, thereby enriching interaction scenarios and meeting user usage needs.

[0051] It should be understood that a multi-screen collaboration scenario may include two or more electronic devices that support 2D display, 3D display, or 4D or higher display. In response to the user's operation of a second electronic device (such as a remote control handle, etc.), different electronic devices can display corresponding display images through the interaction of display information to meet the user's interaction needs. This embodiment of the application will not be given one by one.

[0052] In a second aspect, a first electronic device is provided. The first electronic device includes: a processor, a memory, and a display screen. The memory and the display screen are coupled to the processor. The memory is used to store computer program code. The computer program code includes computer instructions. When the processor reads the computer instructions from the memory, the first electronic device executes: obtaining a first positioning result of the second electronic device, where the first positioning result is determined by a wireless positioning signal. A three-dimensional image is displayed, where the three-dimensional image includes a first virtual element that matches the first positioning result. The three-dimensional image is used for at least one user to view or interact with outside the first electronic device. The first electronic device is fixedly placed in a first position.

[0053] According to a second aspect, a three-dimensional picture is provided for simultaneous viewing by multiple users.

[0054] According to the second aspect, or any implementation of the above second aspect, when the processor reads computer instructions from the memory, it also enables the first electronic device to execute: obtain a second positioning result of the third electronic device, and the three-dimensional picture also includes a second virtual element matching the second positioning result.

[0055] According to the second aspect, or any implementation of the second aspect, the second electronic device and the third electronic device are operated by one or two users.

[0056] According to the second aspect, or any implementation of the second aspect, the first virtual element is at least one of a display component, a controllable component, and a trajectory matching the movement of the second electronic device.

[0057] According to the second aspect, or any implementation of the second aspect above, when the processor reads the computer instructions from the memory, the processor further causes the first electronic device to: receive input data sent by the second electronic device, where the input data includes at least one of key input data, hand motion input data, and joystick operation input data. The input data is used to interact with the three-dimensional image.

[0058] According to the second aspect, or any implementation of the second aspect above, the input data corresponding to the joystick operation is used to instruct to update the display content or display angle of the three-dimensional image.

[0059] According to the second aspect, or any implementation of the second aspect above, when the processor reads the computer instructions from the memory, the processor further causes the first electronic device to execute: obtaining a third positioning result of the second electronic device; and updating the display content or display angle of the three-dimensional image based on the third positioning result.

[0060] According to the second aspect, or any implementation of the second aspect above, when the processor reads computer instructions from the memory, it also enables the first electronic device to execute: based on the first positioning result, determine the three-dimensional picture through the first software in the first electronic device, and the first software is the first virtual reality VR software.

[0061] According to the second aspect, or any implementation of the second aspect above, when the processor reads the computer instructions from the memory, the processor further causes the first electronic device to execute: sending the first positioning result to the fourth electronic device; and receiving a three-dimensional image fed back by the fourth electronic device, where the three-dimensional image is obtained by the fourth electronic device after processing the first positioning result using second software in the fourth electronic device, where the second software is second VR software.

[0062] According to the second aspect, or any implementation of the above second aspect, when the processor reads computer instructions from the memory, it also enables the first electronic device to execute: collecting user images through the camera, and the user images are used to determine at least one parameter of the number of users, user locations, and user physical information.

[0063] According to the second aspect, or any implementation of the second aspect above, when the processor reads the computer instruction from the memory, it also causes the first electronic device to execute: obtaining location information of the first electronic device, and the location information is used for displaying a three-dimensional image.

[0064] According to the second aspect, or any implementation of the second aspect above, the first positioning result includes at least one of the distance between the second electronic device and the first electronic device, the orientation information of the second electronic device, and the height information of the second electronic device.

[0065] According to the second aspect, or any implementation of the second aspect above, a first positioning module is configured in a first electronic device, and a second positioning module is configured in a second electronic device. Obtaining a first positioning result of the second electronic device includes: sending a first wireless positioning signal to the second electronic device via the first positioning module. Receiving the first positioning result sent by the second electronic device, where the first positioning result is determined by the second electronic device via the second positioning module based on the first wireless positioning signal. And / or, receiving, via the first positioning module, a second wireless positioning signal sent by the second electronic device via the second positioning module. Determining the first positioning result via the first positioning module based on the second wireless positioning signal.

[0066] According to the second aspect, or any implementation of the second aspect above, the first positioning module or the second positioning module is any one of the following: an ultra-wideband technology UWB positioning module, an electromagnetic positioning module, a millimeter wave positioning module, or an optical tracking positioning module.

[0067] According to the second aspect, or any implementation of the second aspect above, the three-dimensional picture is used for at least one user to watch outside the first electronic device, including: the three-dimensional picture is used for at least one user to watch while wearing three-dimensional glasses or with naked eyes, and the three-dimensional glasses include shutter glasses or polarized glasses.

[0068] In a third aspect, an electronic device is provided, which has the function of implementing the three-dimensional interactive display method described in the first aspect and any possible implementation thereof. This function can be implemented by hardware, or by hardware executing corresponding software. The hardware or software includes one or more modules corresponding to the above-mentioned functions.

[0069] In a fourth aspect, a computer-readable storage medium is provided. The computer-readable storage medium stores a computer program (also referred to as instructions or code), which, when executed by an electronic device, causes the electronic device to perform the method of the first aspect or any one of the embodiments of the first aspect.

[0070] In a fifth aspect, a computer program product is provided. When the computer program product is run on an electronic device, the electronic device executes the method of the first aspect or any one of the embodiments of the first aspect.

[0071] In a sixth aspect, a circuit system is provided, the circuit system including a processing circuit, the processing circuit being configured to execute the method of the first aspect or any one of the embodiments of the first aspect.

[0072] In the seventh aspect, a chip system is provided, comprising at least one processor and at least one interface circuit, wherein the at least one interface circuit is used to perform transceiver functions and send instructions to the at least one processor. When the at least one processor executes the instructions, the at least one processor executes the method of the first aspect or any one of the embodiments of the first aspect.

[0073] The technical effects of the aforementioned aspects can be referenced with each other and will not be repeated here. BRIEF DESCRIPTION OF THE DRAWINGS

[0074] FIG1 is a first schematic diagram of a display scene of a VR head-mounted display provided in an embodiment of the present application;

[0075] FIG2 is a second schematic diagram of a display scene of a VR head display provided in an embodiment of the present application;

[0076] FIG3 is a schematic diagram of a communication system for applying the three-dimensional interactive display method provided in an embodiment of the present application;

[0077] FIG4 is a schematic diagram of the hardware structure of a first electronic device provided in an embodiment of the present application;

[0078] FIG5 is a schematic diagram of a 3D interactive display scene provided in an embodiment of the present application;

[0079] FIG6 is a second schematic diagram of a 3D interactive display scene provided in an embodiment of the present application;

[0080] FIG7 is a third schematic diagram of a 3D interactive display scene provided in an embodiment of the present application;

[0081] FIG8 is a fourth schematic diagram of a 3D interactive display scene provided in an embodiment of the present application;

[0082] FIG9 is a schematic diagram of the control principle of the joystick provided in an embodiment of the present application;

[0083] FIG10 is a fifth schematic diagram of a 3D interactive display scene provided in an embodiment of the present application;

[0084] FIG11 is a schematic diagram of a flow chart of a three-dimensional interactive display method provided in an embodiment of the present application;

[0085] FIG12 is a schematic structural diagram of a first electronic device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0086] The technical solutions in the embodiments of the present application are described below in conjunction with the drawings in the embodiments of the present application. Among them, in the description of the embodiments of the present application, the terms used in the following embodiments are only for the purpose of describing specific embodiments, and are not intended to be used as limitations on the present application. As used in the specification and appended claims of the present application, the singular expressions "one", "a kind of", "said", "above", "the" and "this" are intended to include expressions such as "one or more", unless there is a clear indication to the contrary in the context. It should also be understood that in the following embodiments of the present application, "at least one", "one or more" refer to one or more (including two).

[0087] References to "one embodiment" or "some embodiments" etc. described in this specification mean that the specific features, structures or characteristics described in conjunction with the embodiment are included in one or more embodiments of the present application. Therefore, the statements "in one embodiment", "in some embodiments", "in some other embodiments", "in some other embodiments", etc. appearing in different places in this specification do not necessarily refer to the same embodiment, but mean "one or more but not all embodiments", unless otherwise specifically emphasized in another way. The terms "including", "comprising", "having" and their variations all mean "including but not limited to", unless otherwise specifically emphasized in another way. The term "connected" includes direct and indirect connections, unless otherwise stated. "First" and "second" are used for descriptive purposes only and are not to be understood as indicating or implying relative importance or implicitly indicating the number of technical features indicated.

[0088] In the embodiments of this application, words such as "exemplarily" or "for example" are used to indicate examples, illustrations, or explanations. Any embodiment or design described as "exemplarily" or "for example" in the embodiments of this application should not be interpreted as being preferred or advantageous over other embodiments or designs. Rather, the use of words such as "exemplarily" or "for example" is intended to present the relevant concepts in a concrete manner.

[0089] In some embodiments, VR devices, based on virtual display technology, can immerse users in a virtual world, enabling user interaction within the virtual environment. For example, by wearing a VR head-mounted display (or helmet), a user can immerse themselves in a virtual 3D gaming environment and interact with virtual elements in the 3D game, thereby enhancing the user's gaming experience.

[0090] In some examples, a VR device includes a VR headset, a controller, and sensors. When a user puts on the VR headset, it provides a 3D visual display and audio playback experience for each eye, immersing the user in the virtual world. The controller, such as a handle or glove, allows the user to interact with the virtual world. For example, the user can control the movement of virtual elements through the controller, and the controller vibrates to provide operational feedback. Sensors track the user's position and movements in the virtual world to achieve a more realistic interactive experience. For example, a VR headset uses a height sensor to determine the headset's height relative to the ground, which is then used as the user's height in subsequent engineering work. Another example is that the VR headset uses a gyroscope and accelerometer to obtain user head movement parameters to adjust the direction and viewing angle of the 3D display to ensure a consistent viewing experience. Interaction with the controller can also determine the controller's direction and distance to the user.

[0091] In some cases, a VR headset is equipped with a battery to operate the VR headset. However, the weight of the battery makes the VR headset heavier, which can easily cause fatigue to the user.

[0092] In some cases, when a VR headset is powered by an external power source, it is necessary to connect an external power cord. However, since users cannot observe their surroundings while wearing the VR headset, they may easily trip over the power cord, which may cause danger.

[0093] In some cases, when using a VR headset, the user is completely immersed in the virtual world, unable to observe the actual surrounding environment. This can easily cause the user to collide with objects in the surrounding environment, affecting user safety. For this reason, before each use of a VR headset, altitude and orientation calibration is required for the user to determine a safe zone to maximize user safety. However, this does not completely reduce user risks, and calibration before each use also increases the difficulty of use.

[0094] For example, as shown in FIG1 , during use, the directional positioning transmitting module in the VR headset sends a positioning signal to the two handles, so that the handles determine the handle positioning result based on the positioning signal. The handle positioning result is the relative position information of the handle relative to the VR headset. Afterwards, the VR headset can obtain the positioning result sent by the handle through the wireless communication module. In addition, the VR headset can obtain the positioning result of the VR headset through a 6-degree-of-freedom (DOF) sensor. Afterwards, the VR headset combines the handle positioning result and the VR headset positioning result through VR software, and outputs a display screen based on the head tracking algorithm and the deformation algorithm, so that the output display screen can be a circular display screen that matches the left and right eye display screens, and can ensure that the screen does not move relative to the head when the user's head moves, thereby providing the user with a better virtual world display effect.

[0095] During the use of the above-mentioned VR headset, the resolution of the VR headset is limited by the resolution of the single-eye display. If the resolution is not high enough, it will cause a screen door effect in the human eye.

[0096] Furthermore, VR headsets rely on head tracking and deformation algorithms for display, requiring high computing power and latency. Insufficient computing power or thermal protection can lead to low frame rates or display freezes, which can cause user discomfort.

[0097] As another example, as shown in FIG2 , a detection base station is arranged near the VR headset and the controller (such as a handle), and the detection base station is, for example, an electronic device such as a PC, a tablet, or a mobile phone. In some examples, the detection base station is configured with a wireless communication module and VR software, and is also configured with a corresponding pointing and positioning transmission module. In this way, the VR headset and the handle can be positioned by the detection base station, and a corresponding display screen can be generated based on the positioning result, and the display screen is sent to the VR headset, which is processed by the VR headset through head tracking algorithms, deformation algorithms, etc. to generate a display screen that meets the user's viewing needs.

[0098] As can be seen, the current solution still suffers from the same issues mentioned above. Furthermore, the detection of the VR headset and controller requires the deployment of additional detection base stations, making the VR headset unable to be used outside of the detection base stations and a fixed environment.

[0099] Therefore, the embodiment of the present application provides a three-dimensional interactive display method, which utilizes a display device equipped with a remote control device, combines the 3D display capability of the display device and the directional remote control capability of the remote control device, and achieves the same 3D virtual display effect as a VR headset without causing the above-mentioned problems, thereby improving the user experience.

[0100] Figure 3 is a schematic diagram of a communication system for the three-dimensional interactive display method provided in an embodiment of the present application. As shown in Figure 3, the communication system includes a first electronic device 100 and a second electronic device 200.

[0101] Optionally, the first electronic device 100 may be a terminal device with 3D display capability, such as a smart screen, a spliced ​​screen device, a tablet computer, a mobile phone, a laptop computer, an ultra-mobile personal computer (UMPC), a netbook, a personal digital assistant (PDA), an artificial intelligence (AI) device, etc. The operating system installed on the first electronic device 100 includes but is not limited to This application does not limit the specific type of the first electronic device 100 or the installed operating system.

[0102] Optionally, the second electronic device 200 may be a terminal device with a directional remote control function, such as a remote control handle, remote control gloves, wearable devices, mobile phones, AI devices, etc. The operating system installed on the second electronic device 200 includes but is not limited to Or other operating systems, or the second electronic device 200 may not have an operating system installed. This application does not limit the specific type of the second electronic device 200, whether an operating system is installed, and the operating system installed when an operating system is installed.

[0103] In some embodiments, a wireless communication connection is established between the first electronic device 100 and the second electronic device 200. The wireless communication technology used to establish the wireless communication connection includes, but is not limited to, at least one of the following: ultra wide band (UWB), Bluetooth (BT) (e.g., traditional Bluetooth or Bluetooth low energy (BLE)), wireless local area networks (WLAN) (e.g., wireless fidelity (Wi-Fi) networks), near field communication (NFC), Zigbee, frequency modulation (FM), infrared (IR), etc.

[0104] Optionally, the first electronic device 100 and the second electronic device 200 may also establish a communication connection through a third-party device in the local area network, such as a router, a gateway, a smart device controller, a server, etc.

[0105] In other embodiments, a wired communication connection may also be established between the first electronic device 100 and the second electronic device 200. For example, the first electronic device 100 and the second electronic device 200 are connected via a universal serial bus (USB) interface.

[0106] Optionally, in a remote control scenario, the first electronic device 100 is, for example, a controlled display terminal (such as a smart screen), and the second electronic device 200 is, for example, a remote control terminal (such as a remote controller). The first electronic device 100 locates the position of the second electronic device 200, combines the positioning result of the second electronic device 200 with the key signal input by the second electronic device 200, and displays the corresponding 3D image to achieve interaction with the user. During this process, the user can watch the 3D display image by wearing 3D glasses or with naked eyes. The user can observe the real environment while entering the 3D virtual world, improving usage safety.

[0107] Optionally, the first electronic device 100 and the second electronic device 200 in the embodiment of the present application may be implemented by different devices, and the different devices may have the same, similar or somewhat different hardware structures, such as the hardware structure shown in FIG. 4 .

[0108] For example, taking the first electronic device 100 having the hardware structure shown in FIG. 4 as an example, the hardware structure shown in FIG. 4 is described.

[0109] As shown in Figure 4, the first electronic device 100 may include a processor 110, an external memory interface 120, an internal memory 121, a universal serial bus (USB) interface 130, a charging management module 140, a power management module 141, a battery 142, an antenna 1, an antenna 2, a mobile communication module 150, a wireless communication module 160, an audio module 170, a sensor module 180, a button 190, a motor 191, an indicator 192, a camera 193, a display screen 194, and a subscriber identification module (SIM) card interface 195, etc.

[0110] It should be understood that the structures illustrated in the embodiments of the present application do not constitute a specific limitation on the first electronic device 100. In other embodiments of the present application, the first electronic device 100 may include more or fewer components than shown, or may combine or separate certain components, or arrange the components differently. The illustrated components may be implemented in hardware, software, or a combination of software and hardware.

[0111] The processor 110 may include one or more processing units. For example, the processor 110 may include an application processor (AP), a modem processor, a graphics processing unit (GPU), an image signal processor (ISP), a controller, a video codec, a digital signal processor (DSP), a baseband processor, and / or a neural-network processing unit (NPU). The different processing units may be independent devices or integrated into one or more processors.

[0112] The controller can generate operation control signals according to the instruction operation code and timing signal to complete the control of instruction fetching and execution.

[0113] Processor 110 may also include a memory for storing instructions and data. In some embodiments, the memory in processor 110 is a cache memory. This memory can store instructions or data that have just been used or are being recycled by processor 110. If processor 110 needs to use the same instruction or data again, it can directly access the memory. This avoids duplicate accesses, reduces processor 110 latency, and thus improves system efficiency.

[0114] In some embodiments, the processor 110 may include one or more interfaces. The interfaces may include an inter-integrated circuit (I2C) interface, an inter-integrated circuit sound (I2S) interface, a pulse code modulation (PCM) interface, a universal asynchronous receiver / transmitter (UART) interface, a mobile industry processor interface (MIPI), a general-purpose input / output (GPIO) interface, a subscriber identity module (SIM) interface, and / or a universal serial bus (USB) interface.

[0115] The MIPI interface can be used to connect the processor 110 to peripheral devices such as the display screen 194 and the camera 193. MIPI interfaces include the camera serial interface (CSI) and the display serial interface (DSI). In some embodiments, the processor 110 and the camera 193 communicate via the CSI interface to implement the camera function of the first electronic device 100. The processor 110 and the display screen 194 communicate via the DSI interface to implement the display function of the first electronic device 100.

[0116] The USB interface 130 is an interface that complies with USB standards and specifications, and may be a Mini USB interface, a Micro USB interface, a USB Type-C interface, or the like. The USB interface 130 can be used to connect a charger to charge the first electronic device 100, or to transfer data between the first electronic device 100 and peripheral devices. It can also be used to connect headphones to play audio. This interface can also be used to connect other electronic devices, such as AR devices.

[0117] It is understood that the interface connection relationship between the modules illustrated in the embodiments of the present application is merely an illustrative description and does not constitute a structural limitation on the first electronic device 100. In other embodiments of the present application, the first electronic device 100 may also adopt a different interface connection method from the above embodiments, or a combination of multiple interface connection methods.

[0118] The charging management module 140 is configured to receive charging input from a charger. The charger can be either a wireless charger or a wired charger. In some wired charging embodiments, the charging management module 140 can receive charging input from the wired charger via the USB interface 130. In some wireless charging embodiments, the charging management module 140 can receive wireless charging input via the wireless charging coil of the first electronic device 100. While charging the battery 142, the charging management module 140 can also provide power to the electronic device via the power management module 141.

[0119] The power management module 141 is used to connect the battery 142, the charging management module 140, and the processor 110. The power management module 141 receives input from the battery 142 and / or the charging management module 140, and provides power to the processor 110, the internal memory 121, the display 194, the camera 193, and the wireless communication module 160. The power management module 141 can also be used to monitor parameters such as battery capacity, battery cycle count, and battery health status (leakage, impedance). In some other embodiments, the power management module 141 can also be set in the processor 110. In other embodiments, the power management module 141 and the charging management module 140 can also be set in the same device.

[0120] The wireless communication function of the first electronic device 100 can be implemented through the antenna 1, the antenna 2, the mobile communication module 150, the wireless communication module 160, the modem processor and the baseband processor.

[0121] Antenna 1 and Antenna 2 are used to transmit and receive electromagnetic wave signals. Each antenna in first electronic device 100 can be used to cover a single or multiple communication frequency bands. Different antennas can also be reused to improve antenna utilization. For example, antenna 1 can be reused as a diversity antenna for a wireless local area network. In other embodiments, the antennas can be used in conjunction with a tuning switch.

[0122] The mobile communication module 150 can provide solutions for wireless communications including 2G / 3G / 4G / 5G applied to the first electronic device 100. The mobile communication module 150 may include at least one filter, a switch, a power amplifier, a low noise amplifier (LNA), etc. The mobile communication module 150 can receive electromagnetic waves from the antenna 1, and filter, amplify, and process the received electromagnetic waves, and transmit them to the modulation and demodulation processor for demodulation. The mobile communication module 150 can also amplify the signal modulated by the modulation and demodulation processor, and convert it into electromagnetic waves for radiation through the antenna 1. In some embodiments, at least some of the functional modules of the mobile communication module 150 can be set in the processor 110. In some embodiments, at least some of the functional modules of the mobile communication module 150 can be set in the same device as at least some of the modules of the processor 110.

[0123] The modem processor may include a modulator and a demodulator. The modulator is used to modulate the low-frequency baseband signal to be transmitted into a medium-high frequency signal. The demodulator is used to demodulate the received electromagnetic wave signal into a low-frequency baseband signal. The demodulator then transmits the demodulated low-frequency baseband signal to the baseband processor for processing. After being processed by the baseband processor, the low-frequency baseband signal is passed to the application processor. The application processor outputs the sound signal through the audio device or displays an image or video through the display screen 194. In some embodiments, the modem processor may be an independent device. In other embodiments, the modem processor may be independent of the processor 110 and be provided in the same device as the mobile communication module 150 or other functional modules.

[0124] The wireless communication module 160 can provide wireless communication solutions including wireless local area networks (WLAN) (such as wireless fidelity (Wi-Fi) networks), Bluetooth (BT), global navigation satellite system (GNSS), frequency modulation (FM), near field communication (NFC), infrared (IR), etc., applied on the first electronic device 100. The wireless communication module 160 can be one or more devices integrating at least one communication processing module. The wireless communication module 160 receives electromagnetic waves via the antenna 2, frequency modulates and filters the electromagnetic wave signals, and sends the processed signals to the processor 110. The wireless communication module 160 can also receive the signal to be sent from the processor 110, frequency modulate it, amplify it, and convert it into electromagnetic waves for radiation through the antenna 2.

[0125] In some embodiments, the antenna 1 of the first electronic device 100 is coupled to the mobile communication module 150, and the antenna 2 is coupled to the wireless communication module 160, so that the first electronic device 100 can communicate with a network and other devices through wireless communication technology. The wireless communication technology may include global system for mobile communications (GSM), general packet radio service (GPRS), code division multiple access (CDMA), wideband code division multiple access (WCDMA), time-division code division multiple access (TD-SCDMA), long term evolution (LTE), BT, GNSS, WLAN, NFC, FM, and / or IR technology. The GNSS may include a global positioning system (GPS), a global navigation satellite system (GLONASS), a Beidou navigation satellite system (BDS), a quasi-zenith satellite system (QZSS) and / or a satellite based augmentation system (SBAS).

[0126] The first electronic device 100 implements display functionality through a GPU, display screen 194, and an application processor. The GPU is a microprocessor for image processing that connects the display screen 194 and the application processor. The GPU is used to perform mathematical and geometric calculations for graphics rendering. The processor 110 may include one or more GPUs that execute program instructions to generate or modify display information.

[0127] The display screen 194 is used to display images, videos, etc. The display screen 194 includes a display panel. The display panel can be manufactured using a liquid crystal display (LCD), for example, an organic light-emitting diode (OLED), an active-matrix organic light-emitting diode or an active-matrix organic light-emitting diode (AMOLED), a flexible light-emitting diode (FLED), Mini-LED, Micro-LED, Micro-OLED, quantum dot light-emitting diodes (QLED), etc. In some embodiments, the first electronic device 100 may include one or N display screens 194, where N is a positive integer greater than 1.

[0128] The camera 193 is used to capture still images or videos. The object generates an optical image through the lens and projects it onto the photosensitive element. The photosensitive element can be a charge coupled device (CCD) or a complementary metal-oxide-semiconductor (CMOS) phototransistor. The photosensitive element converts the light signal into an electrical signal, and then passes the electrical signal to the ISP for conversion into a digital image signal. The ISP outputs the digital image signal to the DSP for processing. The DSP converts the digital image signal into an image signal in a standard RGB, YUV or other format. In some embodiments, the first electronic device 100 may include 1 or N cameras 193, where N is a positive integer greater than 1.

[0129] The external memory interface 120 can be used to connect an external memory card, such as a Micro SD card, to expand the storage capacity of the first electronic device 100. The external memory card communicates with the processor 110 via the external memory interface 120 to implement data storage functions. For example, files such as music and videos can be stored in the external memory card.

[0130] The internal memory 121 can be used to store computer executable program codes, which include instructions. The internal memory 121 may include a program storage area and a data storage area. Among them, the program storage area may store an operating system, an application required for at least one function (such as a sound playback function, an image playback function, etc.), etc. The data storage area may store data created during the use of the first electronic device 100 (such as audio data, a phone book, etc.), etc. In addition, the internal memory 121 may include a high-speed random access memory, and may also include a non-volatile memory, such as at least one disk storage device, a flash memory device, a universal flash storage (UFS), etc. The processor 110 executes various functional applications and data processing of the first electronic device 100 by running instructions stored in the internal memory 121 and / or instructions stored in a memory provided in the processor.

[0131] The audio module 170 is used to convert digital audio information into analog audio signal output, and is also used to convert analog audio input into digital audio signals. The audio module 170 can also be used to encode and decode audio signals. In some embodiments, the audio module 170 can be set in the processor 110, or some functional modules of the audio module 170 can be set in the processor 110. The first electronic device 100 can use the audio module 170, such as music playback and recording. The audio module 170 may include a speaker, a receiver, a microphone, a headphone jack, and an application processor to implement audio functions.

[0132] The sensor module 180 may include a pressure sensor, a gyro sensor, an air pressure sensor, a magnetic sensor, an acceleration sensor, a distance sensor, a proximity light sensor, a fingerprint sensor, a temperature sensor, a touch sensor, an ambient light sensor, a bone conduction sensor, and the like.

[0133] The buttons 190 include a power button, a volume button, etc. The buttons 190 may be mechanical buttons or touch buttons. The first electronic device 100 may receive key inputs and generate key signal inputs related to user settings and function control of the first electronic device 100.

[0134] Motor 191 can generate vibration alerts. This can be used for incoming call vibration alerts or touch vibration feedback. Depending on the touch operation applied to different areas of the display 194, motor 191 can also generate different vibration feedback effects. Different application scenarios (e.g., time reminders, receiving messages, alarm clocks, gaming, etc.) can also correspond to different vibration feedback effects.

[0135] The indicator 192 may be an indicator light, which may be used to indicate the charging status, power level changes, messages, missed calls, notifications, etc.

[0136] The SIM card interface 195 is used to connect a SIM card. The SIM card can be connected to and disconnected from the first electronic device 100 by inserting or removing the SIM card into or from the SIM card interface 195. The first electronic device 100 may support one or N SIM card interfaces, where N is a positive integer greater than one.

[0137] The following describes a three-dimensional interactive display method provided in an embodiment of the present application by taking the first electronic device 100 as a smart screen and the second electronic device 200 as a remote control handle as an example.

[0138] In some embodiments, the smart screen can obtain the positioning results of the remote controller and determine the position, height, and other information of the remote controller based on the positioning results. In some examples, the user holds the remote controller to interact with the smart screen. Therefore, the smart screen can determine the user's height, distance from the smart screen, arm length, and other information based on the position and height of the remote controller. Based on this user information, it can provide the user with a 3D image display that better meets the user's needs.

[0139] In some examples, the 3D picture includes a static picture, a video picture, a displayed picture, a picture that can interact with the user, etc. For example, the 3D picture includes multiple types such as a static image or a frame image in a video picture.

[0140] Exemplarily, as shown in FIG5 , the smart screen is configured with two remote control handles, and the two remote control handles establish a communication connection with the smart screen through a wired connection or a wireless connection, and the same user operates the two remote control handles with both hands. In some examples, as shown in FIG5 , the smart screen is configured with a pointing and positioning transmitting module for sending positioning signals to the two remote control handles respectively. Accordingly, after receiving the positioning signal, the remote control handle determines its own positioning result relative to the smart screen based on the positioning signal. Optionally, the positioning result includes information such as the direction of the remote control handle and the distance relative to the smart screen. Afterwards, the remote control handle sends the determined positioning result to the smart screen. Accordingly, as shown in FIG5 , the smart screen receives the positioning result sent by the remote control handle through the wireless communication module.

[0141] It should be understood that generally, the position of the smart screen will not be moved when the user uses the smart screen, so the positioning result of the remote control handle relative to the smart screen obtained by the smart screen can be absolute positioning.

[0142] As another example, the remote controller sends a positioning signal to the smart screen in response to a user operation. Upon receiving the positioning signal, the smart screen can generate a positioning result for the remote controller based on the positioning signal. In other words, the positioning result for the remote controller can be determined by at least one of the remote controller and the smart screen.

[0143] In some examples, a positioning module is configured in the smart screen and / or remote control handle, which can be used to send and / or receive positioning signals to generate positioning results.

[0144] In some embodiments, after the smart screen is started, through the interaction between the smart screen and the remote control handle, the smart screen can complete the initialization process of the remote control handle based on the positioning results obtained one or more times. During the initialization process, the smart screen can complete the positioning of the remote control handle and determine the physical information (such as height, arm length, etc.) of the user currently using the remote control handle. Thus, in the subsequent use process, the smart screen can achieve more accurate interaction with the user. Optionally, the smart screen completes the initialization process according to a preset threshold value for the number of positioning results obtained. Alternatively, after the smart screen determines that the positioning of the remote control handle and the user's physical information have been obtained, it can determine that the initialization process is completed.

[0145] In some examples, during use, the smart screen can determine, based on the positioning results, that the deviation of the user's physical information exceeds a preset threshold and that the current user of the remote controller has changed. The smart screen can then re-initialize the remote controller by obtaining the positioning results. This allows the screen to adaptively adjust usage data to meet user needs during use.

[0146] In some cases, the user of the smart screen generally does not change during use. Therefore, after initialization, the smart screen can no longer obtain the user's physical information, but instead generate a corresponding 3D image display based on the positioning results of the remote controller, thereby saving computing power.

[0147] In some examples, the smart screen can store information such as the user's physical information and the positioning information of the remote control handle. Therefore, when the smart screen triggers the 3D interaction process in response to user operations (such as when the user starts a 3D game, etc.), the smart screen can directly realize the interaction between the 3D screen display and the user based on the historical storage information. In addition, the smart screen can also adaptively correct relevant information during the interaction process. Therefore, after the user turns on the power, there is no need to calibrate the smart screen first, but the smart screen can be used directly for 3D display, thereby reducing the user's waiting. In addition, the smart screen can adaptively adjust relevant information during use to optimize the display effect and better meet the user's usage needs.

[0148] In some examples, the smart screen sends a positioning signal to the remote controller at a preset period, and the corresponding smart screen can obtain the positioning result of the remote controller at the preset period. Optionally, the smart screen can also obtain the positioning result at the preset period through other methods.

[0149] In this way, compared to VR headsets, which require users to manually perform initialization calibration after each power-on and determine the safe zone before use, the three-dimensional interactive display method provided in the embodiments of the present application generally does not require the smart screen to move. Based on the storage and display characteristics of the smart screen, there is no need to force the user to perform the initialization calibration process after powering on, which effectively simplifies user operations and reduces user difficulty.

[0150] In some embodiments, the principles of stereoscopic vision in the human eye include binocular stereo, monocular stereo (static), monocular motion stereo, and light and shadow effects. The human eye has the ability to adjust line of sight (convergence) and adjust the focal length of the lens, which correspond to binocular and monocular stereo. The principle of binocular stereo is that when a user views with both eyes, the extended lines of sight of both eyes intersect at a point, known as the point of convergence. When viewing objects at different distances, the direction of the eye's sight is continuously adjusted. The movement of the eye muscles and the coordinated information are synthesized by the human brain into stereo information with spatial distance. The principle of monocular stereo is that when a user views with one eye, the formation of stereo is mainly related to the focusing of the lens. When viewing objects at different distances, the focus of the lens is continuously adjusted until the image is clear (focused). When a person views objects close up, the lens thickens and when viewing objects far away, it thins. The movement of the eye muscles and the coordinated information are synthesized by the human brain into stereo information with spatial distance. Monocular motion stereo is achieved by subtly adjusting the body or head position when focal point information is insufficient (for example, at long distances and with minimal focus change). This secondary perspective, based on the relationship between obstacles in front and behind the object, provides additional spatial information. Light and shadow effects utilize the physical effects of light and shadow on an object to create a 3D representation of the object within the image.

[0151] Based on the aforementioned principle of human stereoscopic vision, the Smart Screen uses binocular parallax to create a stereoscopic display, achieving 3D image display capabilities. The human eyes are approximately 6.5 cm apart. When viewing an object with both eyes, the left and right eyes each see two slightly different images, and the images formed on the left and right retinas are also different. The combined reaction of the cerebral cortex produces a three-dimensional perception. Therefore, the Smart Screen can generate images for the left and right eyes and input them to the left and right eyes respectively, allowing people to see a three-dimensional image.

[0152] In some examples, the smart screen is integrated with VR software for generating 3D images. Based on the above principles, the VR software can generate 3D images and provide users with a 3D experience of the virtual world.

[0153] In some cases, after obtaining the positioning results of the remote controller, the smart screen sends the positioning results to the VR software. Then, when generating 3D images, the VR software can combine the positioning results to generate 3D images that meet the user's needs.

[0154] In some examples, the remote control handle can receive user operations and send the input data corresponding to the operations to the smart screen. Then, in the process of generating 3D images, the VR software in the smart screen can also combine the input data to generate corresponding 3D images, thereby enabling user interaction with the 3D images. Optionally, the operations include, for example, the user operating the buttons on the remote control handle, the user's hand movements (such as movement operations in different directions), etc.

[0155] Optionally, the Smart Screen displays corresponding virtual elements in the 3D image generated based on the input data and positioning results sent by the remote controller. These virtual elements are used to indicate the user's operation of the remote controller. Optionally, the virtual elements are displayed in the 3D image as, for example, a cursor, a preset graphic, a trajectory, etc. For example, if the user moves the remote controller to the left, the Smart Screen displays the leftward movement trajectory of the remote controller on the display screen based on the changes in the positioning results of the remote controller.

[0156] Optionally, after the remote control handle generates input data in response to user operations, it can also provide feedback on the operation effects to the user through vibration, key force feedback, etc., thereby making the user more immersed in the virtual world.

[0157] For example, as shown in Figure 5, the smart screen integrates VR software. After the wireless communication module receives the positioning results, input data, and other information sent by the remote controller, it sends the positioning results, input data, and other information to the VR software. The VR software then generates a 3D image based on the positioning results, input data, and other information and sends the 3D image to the 3D display module for display.

[0158] In this way, after the smart screen generates a 3D image through VR software, it can be sent directly to the display module for display. There is no need for the image processing process of the VR headset to go through the deformation algorithm and head tracking algorithm, thereby reducing the requirements for computing power and reducing the image processing delay, avoiding the display frame rate reduction or abnormal freezing.

[0159] It should be understood that in addition to providing users with a 3D picture display experience, the smart screen can also provide users with a sound playback experience, allowing users to immerse themselves in the virtual world. This will not be elaborated below.

[0160] In some embodiments, after the smart screen generates a 3D image using VR software, it can display the 3D image. Users can then view the 3D image displayed on the smart screen by wearing 3D glasses. Furthermore, while viewing, users can interact with the 3D image by operating a remote controller.

[0161] Optionally, users can watch the 3D images displayed on the smart screen by wearing glasses that use 3D technology, such as shutter glasses or polarized glasses.

[0162] Shutter glass 3D technology is an active 3D technology. Using a pair of active LCD shutter glasses, the left and right eyes are alternately opened and closed, allowing the user to see two images alternately, which then merge into one in the brain, providing the user with a sense of 3D depth. Shutter glass 3D technology works based on the human eye's ability to adjust image refresh rates. For example, the image refresh rate must be greater than or equal to 120 Hz. Therefore, the left and right eyes refresh the image at a rapid rate of 60 Hz, preventing the user from experiencing jitter and maintaining the same frame rate as 2D video. The user's two glasses see different images rapidly switching, creating an illusion in the brain, resulting in a stereoscopic effect.

[0163] Patterned retarder 3D technology is a passive 3D technology widely used in 3D cinemas, 3D TVs, and other scenarios. For example, a polarizing film applied to the display of a patterned 3D TV allows the image to be transmitted from different directions. Polarized glasses separate the vertical and horizontal images for the left and right eyes, creating a 3D effect by interleaving and overlapping them in the user's brain.

[0164] For example, as shown in FIG5 , the smart screen generates a 3D image through VR software and displays the 3D image through the display screen. The user can watch the 3D image displayed by the smart screen by wearing 3D glasses.

[0165] In this way, users can view and interact with 3D images simply by wearing 3D glasses. Compared to VR headsets, 3D glasses are significantly lighter and provide the same immersive virtual world interaction experience as VR headsets. Furthermore, 3D glasses do not require external power, obstruct the user's vision, or cause discomfort such as heat or stuffiness, thereby effectively improving user safety and comfort.

[0166] In other embodiments, the smart screen is equipped with a 3D display screen. When displaying 3D images generated by VR software on the 3D display screen, users do not need to wear 3D glasses and can directly watch the 3D images with naked eyes. This further reduces the user's operating difficulty and improves the user experience.

[0167] Currently, mainstream naked-eye 3D technologies include slit-type liquid crystal barriers, lenticular lenses, directional light sources, and active backlighting. The slit-type liquid crystal barrier technology employs a slit-type barrier in front of the display screen. When the image is displayed to the left eye, an opaque stripe blocks the view to the right eye. Similarly, when the image is displayed to the right eye, an opaque stripe blocks the view to the left eye. This separates the images visible to the left and right eyes, allowing the user to see a 3D image. The lenticular lens technology utilizes the principle of lens refraction to project the pixels corresponding to the left and right eyes separately, achieving image separation and allowing the user to see a 3D image. The directional light source technology precisely controls two screens to project images to the left and right eyes, respectively, enabling the user to see a 3D image. The active backlight technology utilizes optical microstructures to form a backlight module. The light beam emitted by the backlight can be adjusted in direction under the control of electronic components, allowing the user to see a 3D image.

[0168] In some embodiments, the smart screen establishes a communication connection with an external processing device and generates a 3D image through the VR software installed in the processing device, thereby reducing the computing power requirements of the smart screen and reducing the computing power consumption of the smart screen.

[0169] Optionally, the processing device is a terminal device with 3D image processing capabilities, such as a PC, tablet, or mobile phone. A wired communication connection or a wireless communication connection is established between the smart screen and the processing device. For example, a wired communication connection is established between the smart screen and the processing device via USB, Ethernet, or the like. For another example, a Wi-Fi wireless communication connection is established between the smart screen and the processing device.

[0170] Exemplarily, as shown in FIG6 , a processing device is configured near the smart screen, and VR software is installed in the processing device. The smart screen sends a positioning signal to the remote control handle through the pointing positioning transmitting module. After the remote control handle generates a positioning result based on the received positioning signal, the positioning result is fed back to the smart screen. Afterwards, after the smart screen obtains the positioning result of the remote control handle through the wireless communication module, the positioning result is sent to the adaptation module. The adaptation module is used to forward the obtained positioning result to the processing device. Optionally, the adaptation module is also used to forward the input data sent by the remote control handle received by the smart screen to the processing device. The input data is the data generated by the user operating the remote control handle. Then, the processing device can generate a 3D picture based on the positioning results, input data and other information through the VR software. Optionally, as mentioned above, the positioning result includes information such as the distance between the remote control handle and the smart screen, the direction of the remote control handle, and the input data includes data generated by the user's button operation on the remote control handle, data generated by the user's hand movements while holding the remote control handle, and other data.

[0171] The processing device then sends the generated 3D image to the smart screen. Correspondingly, the smart screen receives the 3D image sent by the processing device through the adaptation module and displays it through the 3D display module. Users can then view the 3D image displayed on the smart screen by wearing 3D glasses or with their naked eyes.

[0172] Optionally, the adaptation module can compress or decompress the data transmitted between the smart screen and the processing device to reduce the data transmission volume and improve data transmission efficiency.

[0173] This reduces the computing power required for smart screens by configuring external processing devices with stronger computing power to acquire 3D images. Furthermore, smart screens display 3D images directly after acquisition, without requiring processing via deformation and head tracking algorithms. This reduces latency and resource requirements, further minimizing the risk of lag.

[0174] Moreover, in the current solution, there is no need to force the user to perform the initialization calibration process after powering on the device, which effectively simplifies user operations and reduces user difficulty.

[0175] In addition, in the current solution, users can wear 3D glasses or watch 3D images with naked eyes, which can reduce the wearing weight, will not block the line of sight, 3D glasses do not require external power supply, and will not cause heat or stuffiness.

[0176] It should be understood that the implementation methods, principles, and effects of the 3D interactive display scene shown in Figure 6 and the multiple 3D interactive display scenes described below can refer to the relevant content described in the 3D interactive display scene shown in Figure 5 above, and will not be repeated here. For example, the positioning result of the remote control handle can also be determined by the smart screen based on the positioning signal sent by the remote control handle.

[0177] In some embodiments, the emergence of multiplayer 3D games meets the needs of users for multiplayer game interaction. However, due to the single-person wearable characteristics of VR headsets, if users need to play multiplayer 3D games, multiple users need to wear their own VR headsets and perform initialization calibration after powering on before they can play the game. In the three-dimensional interactive display method provided in the embodiments of the present application, by integrating the functions of the smart screen and the remote control handle, multiple users only need to wear different 3D glasses or directly use naked eyes to watch the same 3D picture displayed by the smart screen, thereby realizing 3D picture sharing. Moreover, different users can control the virtual elements of the remote control handles corresponding to different users in the 3D picture by operating different remote control handles.

[0178] For example, as shown in Figure 7, user 1 and user 2 need to play a 3D game, then user 1 can hold remote control handle 1 and user 2 can hold remote control handle 2. Afterwards, the smart screen can determine the positioning results of remote control handle 1 and remote control handle 2 by sending positioning signals to remote control handle 1 and remote control handle 2. In addition, the smart screen can receive input data sent by remote control handle 1 and remote control handle 2 respectively. Afterwards, the smart screen can generate a 3D picture through VR software based on the positioning results and input data and display it. In this way, user 1 and user 2 can watch the 3D picture displayed by the smart screen together by wearing 3D glasses or naked eyes, and can operate remote control handle 1 and remote control handle 2 respectively to interact with the smart screen.

[0179] It should be understood that the embodiment of the present application does not limit the number of remote control handles. For example, the user can configure a corresponding number of remote control handles according to needs.

[0180] In addition, the above method is also applicable to 3D display scenarios involving multiple people other than multiplayer 3D games.

[0181] In this way, multiple users can watch the same 3D picture by wearing 3D glasses or with naked eyes, sharing the same picture, and can operate the same VR software through different remote control handles to achieve a multi-person interactive 3D virtual world experience.

[0182] Furthermore, in multi-person 3D interactive scenarios, users do not need to purchase and wear multiple VR headsets, reducing costs while still providing the same user experience. This eliminates the common shortcomings of VR headsets.

[0183] In some embodiments, a 3D virtual world can provide users with a three-dimensional viewing experience. In VR headsets, the headset can detect the user's head movement, generate inversely moving images through VR software, and simulate a static perspective, providing the user with a viewing experience where the image moves with the user's head movement. However, even small head movements can easily lead to unnecessary image adjustments, which not only consumes system resources but can also cause user dizziness if the image adjustments are not timely.

[0184] Based on this, the three-dimensional interactive display method provided in the embodiment of the present application meets the user's needs for adjusting the direction of the three-dimensional picture by adjusting the joystick or direction adjustment button (or touchpad) on the remote control handle and the change of field of view caused by the virtual person's head movement.

[0185] Exemplarily, as shown in FIG8 , a joystick and a joystick detection module for detecting the movement of the joystick are configured on the remote control handle. In some examples, the joystick can shake back and forth or left and right, or can rotate 360 ​​degrees and shake, etc. in response to user operations. The joystick detection module can convert the detected joystick shaking into input data. After that, the remote control handle can send the input data to the smart screen. After receiving the input data through the wireless communication module, the smart screen sends the input data to the VR software, and the VR software can convert the input data into the movement data of the user's head. Then, the VR software can generate a corresponding 3D picture of the field of view change caused by the user's head movement based on the movement data of the user's head, and send it for display. In this way, the user can see the 3D picture adjusted by the joystick through 3D glasses or naked eyes, and realize interaction with the picture displayed by the smart screen.

[0186] For example, as shown in Figure 9, the joystick detection module detects that the user shakes the joystick left or right, generates corresponding input data, and sends it to the smart screen. The smart screen converts the input data into pitch or tilt motion data of the user's head through VR software. For another example, as shown in Figure 9, the joystick detection module detects that the user shakes the joystick forward or backward, generates corresponding input data, and sends it to the smart screen. The smart screen converts the input data into rotational motion data of the user's head through VR software.

[0187] By converting the forward and backward or left and right inputs of the joystick through VR software, it can be equivalent to 6DOF detection for the VR headset. This allows the user to change the viewing angle using only the joystick without moving their head, thus avoiding the feeling of sickness caused by low or dropped frames.

[0188] Furthermore, adjusting the user's perspective based on joystick movement eliminates the need to generate inverse motion images, thereby reducing system resource usage, placing lower demands on the processor, and minimizing the likelihood of frame drops. Furthermore, the perspective changes are smoother.

[0189] In some embodiments, some smart screens are equipped with a camera that can be used to capture user images. Then, in a 3D display scene, the smart screen can capture user images through the camera. Afterwards, the smart screen can determine the user positioning results based on the collected user images, such as the number of users, user locations, and user physical information (such as height, arm length, etc.). Based on the user positioning results and the positioning results of the remote control handle, the smart screen can more accurately determine the display position of the elements that need to be responded to in the 3D picture.

[0190] For example, the remote control handle is displayed as a virtual element in the 3D picture. In response to the user moving the position of the remote control handle, the smart screen can change the display position of the virtual element in the 3D picture, thereby realizing interaction with the user.

[0191] As mentioned above, the Smart Screen can complete the initialization process based on one or more positioning results of the remote controller. Therefore, the Smart Screen can effectively improve the initialization efficiency by combining the user positioning results.

[0192] For example, as shown in Figure 10, the Smart Screen is equipped with a calibration camera that captures the user's image and determines the user's location. The calibration camera then transmits this user location to the VR software, which combines the user's location, the remote controller's location, and the remote controller's input data to generate and display a 3D image. This allows users to view the 3D image with or without 3D glasses, ensuring that the resulting 3D image is more fully responsive to user needs.

[0193] In this way, the initialization of the smart screen is completed through the camera, and the user does not need to participate in the initialization every time the screen is turned on. While ensuring the accurate effect of 3D display, the difficulty of user operation is reduced.

[0194] Figure 11 is a flow chart of a three-dimensional interactive display method provided in an embodiment of the present application. It should be noted that the method is not limited to Figure 11 and the specific sequence described below. It should be understood that in other embodiments, the order of some steps in the method can be interchanged according to actual needs, or some steps can be omitted or deleted. The method includes the following steps:

[0195] S1101: A first electronic device obtains a first positioning result of a second electronic device, where the first positioning result is determined through a wireless positioning signal.

[0196] The first positioning result includes at least one of the distance between the second electronic device and the first electronic device, the direction information of the second electronic device, and the height information of the second electronic device.

[0197] In some examples, the first electronic device is fixedly placed at a first location. In some examples, a spatial coordinate system is established based on the location of the first electronic device, and three-dimensional coordinates and orientation information of the second electronic device relative to the first electronic device are obtained. For example, the three-dimensional coordinates and orientation information are the first positioning result of the second electronic device.

[0198] In some embodiments, the first positioning result is determined by the first electronic device based on the wireless positioning signal sent by the second electronic device. Alternatively, the first positioning result is determined by the second electronic device based on the wireless positioning signal sent by the first electronic device.

[0199] In some examples, a first positioning module is configured in a first electronic device, and a second positioning module is configured in a second electronic device. The first electronic device transmits a first wireless positioning signal to the second electronic device via the first positioning module. The first electronic device then receives a first positioning result sent by the second electronic device, where the first positioning result is determined by the second electronic device via the second positioning module based on the first wireless positioning signal. Alternatively, the first electronic device receives a second wireless positioning signal sent by the second electronic device via the second positioning module via the first positioning module. The first electronic device then determines the first positioning result based on the second wireless positioning signal via the first positioning module.

[0200] The first positioning module or the second positioning module is any one of the following: a UWB positioning module, an electromagnetic positioning module, a millimeter wave positioning module, and an optical tracking positioning module.

[0201] Exemplarily, as shown in Figure 5, the smart screen (such as the first electronic device) establishes a communication connection with the remote control handle (such as the second electronic device) through a wired connection or a wireless connection. The first electronic device is equipped with a pointing positioning transmission module (i.e., a first positioning module) for sending a wireless positioning signal to the second electronic device. Accordingly, after the second electronic device receives the wireless positioning signal, it determines its first positioning result relative to the first electronic device based on the wireless positioning signal. Accordingly, as shown in Figure 5, the first electronic device receives the first positioning result sent by the second electronic device through the wireless communication module.

[0202] In this way, the first electronic device can achieve more accurate positioning of the second electronic device through the wireless positioning signal, so as to subsequently display a three-dimensional image that better meets the user's needs.

[0203] In some embodiments, before obtaining the first positioning result of the second electronic device, the first electronic device further obtains the location information of the first electronic device, and the location information is used for displaying a three-dimensional image.

[0204] For example, when the first electronic device first launches VR software on the first electronic device, it obtains its own position information to initialize the VR software and adjust the viewing angle height of the user using the second electronic device in the three-dimensional image. Optionally, the first electronic device can save this position information so that the display of the three-dimensional image can be adjusted based on this position information during the subsequent three-dimensional image display process.

[0205] In this way, compared to VR headsets that require users to manually perform initialization calibration after each power-on and determine the safe zone before use, the three-dimensional interactive display method provided in the embodiments of the present application generally does not require the first electronic device to move. Based on the storage and display characteristics of the first electronic device, it is not necessary to force the user to perform the initialization calibration process after powering on, effectively simplifying user operations and reducing user difficulty.

[0206] S1102: The first electronic device displays a three-dimensional picture, where the three-dimensional picture includes a first virtual element matching the first positioning result, and the three-dimensional picture is used for at least one user to view or interact with outside the first electronic device.

[0207] In some embodiments, the first electronic device displays a first virtual element matching the first positioning result in a three-dimensional image based on the first positioning result, thereby enabling user interaction with the three-dimensional image through the second electronic device. For example, the movement direction of the first virtual element is related to the directional information of the second electronic device. For another example, the display position of the first virtual element in the three-dimensional image is related to the movement position of the second electronic device operated by the user.

[0208] In some examples, the 3D picture includes a static picture, a video picture, a displayed picture, a picture that can interact with the user, etc. For example, the 3D picture includes multiple types such as a static image or a frame image in a video picture.

[0209] In some embodiments, the first virtual element is at least one of a display component, a controllable component, and a trajectory that matches the movement of the second electronic device.

[0210] For example, the first electronic device displays the first virtual element, i.e., the first virtual element is a display component. For another example, the first electronic device controls changes in the first virtual element in a three-dimensional image, such as changes in display position, size, shape, etc., in response to a user's operation on a second electronic device. In this case, the first virtual element is a controllable component. For another example, the first electronic device displays a movement trajectory of the first virtual element in the process of moving the display position of the first virtual element in a three-dimensional image in response to a user's operation on a second electronic device. In this case, the first virtual element is a trajectory that matches the movement of the second electronic device.

[0211] In some embodiments, the 3D image is for the user to view outside the first electronic device, including: the 3D image is for the user to view wearing 3D glasses or viewing with naked eyes, and the 3D glasses include shutter glasses or polarized glasses.

[0212] For example, as shown in FIG5 , a first electronic device (such as a smart screen) generates a three-dimensional image through VR software and displays the three-dimensional image through a display screen. The user can watch the three-dimensional image displayed on the smart screen by wearing three-dimensional glasses.

[0213] In this way, users can view and interact with 3D images simply by wearing 3D glasses. Compared to VR headsets, 3D glasses are significantly lighter and offer the same immersive virtual world experience. Furthermore, 3D glasses do not require an external power supply, which would otherwise cause discomfort such as heat and stuffiness, thus effectively improving user comfort.

[0214] In addition, when users wear 3D glasses or watch 3D images with naked eyes, their vision will not be blocked. Users can observe the real environment around them at the same time, thereby effectively improving user safety.

[0215] In some embodiments, the 3D image is intended for simultaneous viewing by multiple users.

[0216] In some embodiments, the first electronic device may further obtain a second positioning result of the third electronic device, and the three-dimensional image may further include a second virtual element matching the second positioning result.

[0217] In some examples, the second electronic device and the third electronic device are operated by one or two users. For example, the second electronic device and the third electronic device are a pair of remote control handles, and the user can operate the two remote control handles with his left and right hands respectively. Then, the first virtual element and the second virtual element are the same virtual elements corresponding to the user. For another example, the second electronic device and the third electronic device are a pair of remote control handles, and two users operate the two remote control handles respectively. Then, the first virtual element and the second virtual element are different virtual elements corresponding to the two users respectively. It should be understood that the first electronic device can also be connected to more devices, so that more users can interact with the same three-dimensional picture displayed by the first electronic device by operating different devices. For example, the user can configure a corresponding number of remote control handles according to needs.

[0218] For example, as shown in Figure 7, user 1 and user 2 need to play 3D games, then user 1 can hold remote control handle 1 (such as the second electronic device), and user 2 can hold remote control handle 2 (such as the third electronic device). Afterwards, the smart screen (such as the first electronic device) can determine the positioning results of remote control handle 1 and remote control handle 2 by sending positioning signals to remote control handle 1 and remote control handle 2. In addition, the smart screen can receive input data sent by remote control handle 1 and remote control handle 2 respectively. Afterwards, the smart screen can generate a 3D picture through VR software based on the positioning results and input data and display it. In this way, user 1 and user 2 can watch the 3D picture displayed on the smart screen together by wearing 3D glasses or naked eyes, and can operate the remote control handle 1 and remote control handle 2 respectively to interact with the smart screen.

[0219] In this way, multiple users can watch the same three-dimensional picture by wearing three-dimensional glasses or with naked eyes, sharing the same picture, and can operate the same VR software through different devices to achieve a multi-person interactive three-dimensional virtual world experience.

[0220] Furthermore, in multi-person 3D interactive scenarios, users do not need to purchase and wear multiple VR headsets, reducing costs while still providing the same user experience. This eliminates the common shortcomings of VR headsets.

[0221] In some embodiments, a first electronic device receives input data sent by a second electronic device. The input data includes at least one of key input data, hand motion input data, and joystick operation input data. The input data is used to interact with a three-dimensional image.

[0222] In this way, the first electronic device updates the display of the three-dimensional picture according to the input data, thereby realizing the interaction between the user and the three-dimensional picture.

[0223] In some examples, the input data corresponding to the joystick operation is used to instruct an update of the display content or display angle of the three-dimensional image.

[0224] Exemplarily, as shown in FIG8 , a joystick and a joystick detection module for detecting the movement of the joystick are configured on the remote control handle (such as the second electronic device). In some examples, the joystick can shake back and forth or left and right, or can rotate 360 ​​degrees in response to user operations. The joystick detection module can convert the detected joystick shaking into input data. Afterwards, the remote control handle can send the input data to the smart screen (such as the first electronic device). After receiving the input data through the wireless communication module, the smart screen sends the input data to the VR software, and the VR software can convert the input data into motion data of the user's head. Then, the VR software can generate a corresponding 3D picture of the field of view change caused by the user's head movement based on the motion data of the user's head, and send it for display. In this way, the user can see the 3D picture adjusted by the joystick through 3D glasses or naked eyes, and interact with the picture displayed by the smart screen.

[0225] For example, as shown in Figure 9, input data generated by a user shaking the joystick left or right can be converted into pitch or tilt motion data of the user's head. Input data generated by a user shaking the joystick forward or backward can be converted into rotational motion data of the user's head. This allows the joystick to update the display content or display angle of a 3D image.

[0226] By converting the forward and backward or left and right inputs of the joystick through VR software, it can be equivalent to 6DOF detection for the VR headset. This allows the user to change the viewing angle using only the joystick without moving their head, thus avoiding the feeling of sickness caused by low or dropped frames.

[0227] Furthermore, adjusting the user's perspective based on joystick movement eliminates the need to generate inverse motion images, thereby reducing system resource usage, placing lower demands on the processor, and minimizing the likelihood of frame drops. Furthermore, the perspective changes are smoother.

[0228] In some examples, the second electronic device is a device equipped with a display screen, such as a mobile phone, tablet, etc. Then, the second electronic device can receive user operations by displaying virtual buttons and generate corresponding input data. For example, the second electronic device detects the user's operation of the up, down, left, and right controls displayed on the display screen and generates corresponding input data, which can also be converted into pitch or tilt motion data of the user's head. Thus, the second electronic device enables the user to interact with the three-dimensional picture displayed by the first electronic device through the display of virtual buttons. In some examples, the second electronic device determines the direction and position of the user moving the second electronic device through sensor data and can generate corresponding positioning results.

[0229] In some other embodiments, the first electronic device obtains a third positioning result of the second electronic device, and then updates the display content or display angle of the three-dimensional image based on the third positioning result.

[0230] In this way, in response to changes in the positioning result of the second electronic device, the first electronic device can also update the display of the three-dimensional image accordingly, thereby enabling interaction between the user and the three-dimensional image.

[0231] In some embodiments, the first electronic device captures user images via a camera, wherein the user images are used to determine at least one parameter of the number of users, user locations, and user physical information.

[0232] For example, as shown in FIG10 , a calibration camera is configured in the smart screen (e.g., the first electronic device) to capture user images and determine user positioning results. The calibration camera then sends the user positioning results to the VR software, which combines the user positioning results, the remote controller positioning results, and the input data from the remote controller to generate a three-dimensional image and display it. In this way, users can view the three-dimensional image by wearing three-dimensional glasses or with their naked eyes, and the final three-dimensional image display can better meet the user's needs.

[0233] In this way, the initialization of the first electronic device is completed through image acquisition of the first electronic device, and the user does not need to participate in the initialization every time after starting the device. While ensuring the accurate effect of the three-dimensional display, the user's operation difficulty is reduced.

[0234] In some embodiments, the first electronic device determines the three-dimensional image based on the first positioning result using first software in the first electronic device, where the first software is first VR software. That is, the first electronic device generates the three-dimensional image using the VR software configured therein.

[0235] For example, as shown in Figure 5, a smart screen (e.g., the first electronic device) is integrated with VR software. After receiving positioning results, input data, and other information from a remote controller (e.g., the second electronic device), the wireless communication module sends these information to the VR software. The VR software then generates a three-dimensional image based on the positioning results, input data, and other information, and sends this image to the 3D display module for display.

[0236] In this way, after the first electronic device generates a three-dimensional image through VR software, it can be sent directly to the display module for display. There is no need for the image processing process of the VR headset to go through the deformation algorithm and head tracking algorithm, thereby reducing the requirements for computing power and reducing the image processing delay, avoiding the display frame rate reduction or abnormal freezing.

[0237] In other embodiments, the first electronic device sends a first positioning result to a fourth electronic device. The first electronic device then receives a three-dimensional image from the fourth electronic device. The three-dimensional image is obtained by the fourth electronic device after processing the first positioning result using second software in the fourth electronic device, where the second software is second VR software.

[0238] Exemplarily, as shown in FIG6 , a processing device (such as a fourth electronic device) is configured near the smart screen (such as the first electronic device), and VR software is installed in the processing device. The smart screen sends a positioning signal to the remote control handle through the pointing positioning transmitting module. After the remote control handle generates a positioning result based on the received positioning signal, it feeds back the positioning result to the smart screen. Afterwards, after the smart screen obtains the positioning result of the remote control handle through the wireless communication module, it sends the positioning result to the adaptation module, which is used to forward the obtained positioning result to the processing device. Then, the processing device can generate a 3D picture based on the positioning result, input data and other information through the VR software. Afterwards, the processing device can send the generated three-dimensional picture to the smart screen. Correspondingly, after the smart screen receives the three-dimensional picture sent by the processing device through the adaptation module, it displays the three-dimensional picture through the 3D display module. Then, the user can watch the three-dimensional picture displayed on the smart screen by wearing three-dimensional glasses or naked eyes.

[0239] In some examples, the first VR software and the second VR software are the same VR software. Alternatively, the first VR software and the second VR software are different VR software adapted for different device types. Alternatively, the first VR software and the second VR software are different versions of VR software.

[0240] By configuring a fourth electronic device with greater computing power to capture 3D images, the computing power requirements of the first electronic device are reduced. Furthermore, the first electronic device can directly display the captured 3D images without having to go through the deformation and head tracking algorithms, thus reducing latency and resource requirements and further minimizing the risk of lag.

[0241] Optionally, the first electronic device can also execute the steps and functions performed by the smart screen in the above embodiments, and the second electronic device can also execute the steps and functions of the remote control handle in the above embodiments, thereby realizing the three-dimensional interactive display method provided by the above embodiments.

[0242] In some embodiments, the 3D interactive display method provided by the embodiments of the present application can also be applied to a spliced ​​screen scenario, where the first electronic device is a spliced ​​screen device, the display screen of the first electronic device includes multiple spliced ​​screens, and the first electronic device displays a spliced ​​3D image based on the positioning result of the second electronic device.

[0243] In some examples, a user can interact with the spliced ​​3D image displayed by the first electronic device through a second electronic device. For example, the user can control the movement of a first virtual element from one spliced ​​screen to another spliced ​​screen by moving the second electronic device. In another example, when the user controls the movement of the first virtual element through the second electronic device, the movement trajectory covers multiple spliced ​​screens, and the first electronic device displays the spliced ​​movement trajectory on the multiple spliced ​​screens.

[0244] In some embodiments, the three-dimensional interactive display method provided in the embodiments of the present application can also be applied to multi-screen collaborative scenarios. Among them, the first electronic device can establish a communication connection with other electronic devices with display functions and realize display interaction. For example, the first electronic device is electronic device 1, and the electronic device that performs display interaction with the first electronic device is electronic device 2. Among them, electronic device 1 supports 3D display, and electronic device 2 supports 2D display or 3D display.

[0245] In some examples, while electronic device 1 is displaying a three-dimensional picture 1, in response to a user moving a second electronic device toward a first virtual element of electronic device 2, indicating an operation to display the first virtual element on electronic device 2, electronic device 1 may send display information of the three-dimensional picture 1 to electronic device 2.

[0246] If electronic device 2 supports 3D display, electronic device 2 can display 3D image 2 based on the display information of 3D image 1, and 3D image 2 includes a first virtual element. 3D image 2 is the same as or corresponds to 3D image 1, thereby achieving continuous display of the displayed images. In this way, electronic device 1 enables 3D image interaction between different electronic devices based on user operations, further enriching user interaction needs. For example, while the first virtual element is moving, electronic device 2 is being operated by another user. In this way, the 3D image 2 displayed by electronic device 2 may include the first virtual element and another virtual element corresponding to the other user.

[0247] If electronic device 2 supports 2D display, it can display a 2D image based on the display information of 3D image 1. This 2D image includes a 2D virtual element corresponding to the first virtual element. The 2D image corresponds to 3D image 1. In this way, electronic device 1 enables interactive display between different electronic devices based on user operations, further enriching user interaction needs.

[0248] In other examples, electronic device 2 displays a two-dimensional image that includes a two-dimensional virtual element corresponding to user 1. In response to a user operation, electronic device 2 determines to move the two-dimensional virtual element to electronic device 1 for display, and electronic device 2 may send display information of the two-dimensional image to electronic device 1. Accordingly, after receiving the display information of the two-dimensional image, electronic device 1 may display a corresponding three-dimensional image based on the display information, which includes the three-dimensional virtual element corresponding to the two-dimensional virtual element.

[0249] In this way, when electronic devices support 2D display and / or 3D display, different electronic devices can respond to user operations and realize interaction of display screens, thereby enriching interaction scenarios and meeting user usage needs.

[0250] It should be understood that a multi-screen collaboration scenario may include two or more electronic devices that support 2D display, 3D display, or 4D or higher display. In response to the user's operation of a second electronic device (such as a remote control handle, etc.), different electronic devices can display corresponding display images through the interaction of display information to meet the user's interaction needs. This embodiment of the application will not be given one by one.

[0251] The three-dimensional interactive display method provided by the embodiment of the present application is described in detail above with reference to Figures 5 to 11. The first electronic device provided by the embodiment of the present application is described in detail below with reference to Figure 12.

[0252] In one possible design, Figure 12 is a schematic diagram of the structure of a first electronic device provided in an embodiment of the present application. As shown in Figure 12, the first electronic device 1200 may include: a transceiver unit 1201, a processing unit 1202, and a display unit 1203. The first electronic device 1200 may be used to implement the functions of the first electronic device (such as a smart screen) involved in the above method embodiments.

[0253] Optionally, the transceiver unit 1201 is configured to support the first electronic device 1200 in executing S1101 in FIG. 11 .

[0254] Optionally, the processing unit 1202 is configured to support the first electronic device 1200 in executing S1101 in FIG. 11 .

[0255] Optionally, the display unit 1203 is configured to support the first electronic device 1200 in executing S1102 in FIG. 11 .

[0256] The transceiver unit may include a receiving unit and a transmitting unit, and may be implemented by a transceiver or a transceiver-related circuit component, and may be a transceiver or a transceiver module. The operations and / or functions of each unit in the first electronic device 1200 are respectively for implementing the corresponding processes of the three-dimensional interactive display method described in the above method embodiment. All relevant contents of each step involved in the above method embodiment can be referred to the functional description of the corresponding functional unit. For the sake of brevity, they are not repeated here.

[0257] Optionally, the first electronic device 1200 shown in FIG12 may further include a storage unit (not shown in FIG12 ) storing a program or instruction. When the transceiver unit 1201, the processing unit 1202, and the display unit 1203 execute the program or instruction, the first electronic device 1200 shown in FIG12 may execute the three-dimensional interactive display method described in the above method embodiment.

[0258] The technical effects of the first electronic device 1200 shown in FIG12 may refer to the technical effects of the three-dimensional interactive display method described in the above method embodiment, and will not be repeated here.

[0259] In addition to being in the form of the first electronic device 1200, the technical solution provided in this application may also be a functional unit or chip in the first electronic device, or a device used in conjunction with the first electronic device.

[0260] An embodiment of the present application also provides a chip system, including: a processor, the processor is coupled to a memory, the memory is used to store programs or instructions, when the program or instructions are executed by the processor, the chip system implements the method in any of the above method embodiments.

[0261] Optionally, there may be one or more processors in the chip system. The processor may be implemented in hardware or software. When implemented in hardware, the processor may be a logic circuit, an integrated circuit, etc. When implemented in software, the processor may be a general-purpose processor implemented by reading software code stored in a memory.

[0262] Optionally, the memory in the chip system may be one or more. The memory may be integrated with the processor or may be provided separately from the processor, which is not limited in the embodiments of the present application. For example, the memory may be a non-transient processor, such as a read-only memory (ROM), which may be integrated with the processor on the same chip or provided on different chips. The embodiments of the present application do not specifically limit the type of memory or the configuration of the memory and the processor.

[0263] Exemplarily, the chip system may be a field programmable gate array (FPGA), an application specific integrated circuit (ASIC), a system on chip (SoC), a central processor unit (CPU), a network processor (NP), a digital signal processor (DSP), a microcontroller unit (MCU), a programmable logic device (PLD) or other integrated chips.

[0264] It should be understood that each step in the above method embodiment can be completed by hardware integrated logic circuits in a processor or by software instructions. The method steps disclosed in the embodiments of the present application can be directly embodied as being executed by a hardware processor, or by a combination of hardware and software modules in a processor.

[0265] An embodiment of the present application also provides a computer-readable storage medium, which stores a computer program. When the computer program runs on a computer, the computer executes the above-mentioned related steps to implement the three-dimensional interactive display method in the above-mentioned embodiment.

[0266] An embodiment of the present application further provides a computer program product. When the computer program product is run on a computer, the computer is caused to execute the above-mentioned related steps to implement the three-dimensional interactive display method in the above-mentioned embodiment.

[0267] In addition, embodiments of the present application further provide a device. Specifically, the device may be a component or module, and may include one or more processors and a memory connected to each other. The memory is used to store a computer program. When the computer program is executed by one or more processors, the device performs the three-dimensional interactive display method described in each of the above method embodiments.

[0268] The apparatus, computer-readable storage medium, computer program product, or chip provided in the embodiments of the present application are all used to execute the corresponding methods provided above. Therefore, the beneficial effects that can be achieved can refer to the beneficial effects of the corresponding methods provided above, and will not be repeated here.

[0269] The steps of the method or algorithm described in conjunction with the disclosure of the embodiments of the present application can be implemented in a hardware manner or can be implemented by a processor executing software instructions. The software instructions can be composed of corresponding software modules, and the software modules can be stored in a random access memory (RAM), a flash memory, a read-only memory (ROM), an erasable programmable read-only memory (EPROM), an electrically erasable programmable read-only memory (EEPROM), a register, a hard disk, a mobile hard disk, a read-only compact disc (CD-ROM) or any other form of storage medium known in the art. An exemplary storage medium is coupled to a processor so that the processor can read information from the storage medium and can write information to the storage medium. Of course, the storage medium can also be a component of the processor. The processor and the storage medium can be located in an application specific integrated circuit (ASIC).

[0270] Through the description of the above embodiments, those skilled in the art will clearly understand that for the sake of convenience and brevity, only the division of the above-mentioned functional modules is used as an example. In actual applications, the above-mentioned functions can be assigned to different functional modules as needed; that is, the internal structure of the device can be divided into different functional modules to complete all or part of the functions described above. The specific working processes of the above-mentioned systems, devices, and units can refer to the corresponding processes in the aforementioned method embodiments and will not be repeated here.

[0271] In the several embodiments provided in this application, it should be understood that the disclosed methods can be implemented in other ways. The device embodiments described above are merely illustrative. For example, the division of the modules or units is only a logical function division, and there may be other division methods in actual implementation; for example, multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. In addition, the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, indirect coupling or communication connection of modules or units, which can be electrical, mechanical or other forms.

[0272] In addition, the functional units in the various embodiments of the present application may be integrated into a single processing unit, or each unit may exist physically separately, or two or more units may be integrated into a single unit. The aforementioned integrated units may be implemented in the form of hardware or software functional units.

[0273] Computer-readable storage media include, but are not limited to, any of the following: a USB flash drive, a mobile hard disk, a read-only memory (ROM), a random access memory (RAM), a magnetic disk, or an optical disk, and other media that can store program code.

[0274] The above is only a specific embodiment of the present application, but the scope of protection of this application is not limited to this. Any changes or substitutions within the technical scope disclosed in this application should be included in the scope of protection of this application. Therefore, the scope of protection of this application should be based on the scope of protection of the claims.

Claims

1. A three-dimensional interactive display method, characterized in that: Applied to a first electronic device, the first electronic device is fixedly placed at a first position, the method includes: Acquire a first positioning result of the second electronic device, where the first positioning result is determined by a wireless positioning signal; A three-dimensional picture is displayed, wherein the three-dimensional picture includes a first virtual element matching the first positioning result, and the three-dimensional picture is used for at least one user to watch or interact outside the first electronic device.

2. The method according to claim 1, characterized in that The three-dimensional image is used for multiple users to watch simultaneously.

3. The method according to claim 1 or 2, characterized in that: The method further comprises: A second positioning result of the third electronic device is obtained, and the three-dimensional picture also includes a second virtual element matching the second positioning result.

4. The method according to claim 3, characterized in that The second electronic device and the third electronic device are operated by one or two users.

5. The method according to any one of claims 1 to 4, characterized in that: The first virtual element is at least one of a display component, a controllable component, and a trajectory matching the movement of the second electronic device.

6. The method according to any one of claims 1 to 5, characterized in that: The method further comprises: Receive input data sent by the second electronic device, where the input data includes at least one of key input data, input data corresponding to hand movements, and input data corresponding to joystick operations, and the input data is used to interact with the three-dimensional image.

7. The method according to claim 6, characterized in that The input data corresponding to the joystick operation is used to instruct to update the display content or display angle of the three-dimensional image.

8. The method according to any one of claims 1 to 5, characterized in that: The method further comprises: Obtaining a third positioning result of the second electronic device; According to the third positioning result, the display content or the display angle of the three-dimensional image is updated.

9. The method according to any one of claims 1 to 8, characterized in that: Before displaying the three-dimensional picture, the method further includes: According to the first positioning result, the three-dimensional image is determined by first software in the first electronic device, and the first software is first virtual reality VR software.

10. The method according to any one of claims 1 to 8, characterized in that: Before displaying the three-dimensional picture, the method further includes: Sending the first positioning result to a fourth electronic device; The three-dimensional image fed back by the fourth electronic device is received, where the three-dimensional image is obtained after the fourth electronic device processes the first positioning result through second software in the fourth electronic device, and the second software is second VR software.

11. The method according to any one of claims 1 to 10, characterized in that: After obtaining the first positioning result of the second electronic device, the method further includes: The user image is collected by a camera, and the user image is used to determine at least one parameter of the number of users, the location of users, and the physical information of users.

12. The method according to any one of claims 1 to 11, characterized in that: Before acquiring the first positioning result of the second electronic device, the method further includes: The position information of the first electronic device is acquired, where the position information is used for displaying the three-dimensional image.

13. The method according to any one of claims 1 to 12, characterized in that: The first positioning result includes at least one of the distance between the second electronic device and the first electronic device, the direction information of the second electronic device, and the height information of the second electronic device.

14. The method according to any one of claims 1 to 13, characterized in that: The first electronic device is provided with a first positioning module, the second electronic device is provided with a second positioning module, and the obtaining of a first positioning result of the second electronic device includes: Sending a first wireless positioning signal to the second electronic device through the first positioning module; receiving a first positioning result sent by the second electronic device, where the first positioning result is determined by the second electronic device through the second positioning module according to the first wireless positioning signal; and / or, Receiving, through the first positioning module, a second wireless positioning signal sent by the second electronic device through the second positioning module; The first positioning result is determined by the first positioning module according to the second wireless positioning signal.

15. The method according to claim 14, characterized in that The first positioning module or the second positioning module is any one of the following: an ultra-wideband technology UWB positioning module, an electromagnetic positioning module, a millimeter wave positioning module, and an optical tracking positioning module.

16. The method according to any one of claims 1 to 15, characterized in that: The three-dimensional image is used for at least one user to watch outside the first electronic device, including: the three-dimensional image is used for at least one user to watch wearing three-dimensional glasses or watching with naked eyes, and the three-dimensional glasses include shutter glasses or polarized glasses.

17. An electronic device, characterized in that: include: A processor, a memory and a display screen, wherein the memory and the display screen are coupled to the processor, the memory is used to store computer program code, and the computer program code includes computer instructions. When the processor reads the computer instructions from the memory, the electronic device executes the method as described in any one of claims 1 to 16.

18. A computer-readable storage medium, characterized in that: The computer-readable storage medium comprises a computer program, and when the computer program is executed on an electronic device, the electronic device is enabled to execute the method according to any one of claims 1 to 16.

19. A computer program product, characterized in that When the computer program product is executed on a computer, the computer is caused to execute the method according to any one of claims 1 to 16.

Images