Method and system for displaying 3D objects

Abstract

Aspects described herein relate to methods and systems for displaying 3D objects. The method includes: receiving information identifying a virtual 3D object and first scaling information of the virtual 3D object, wherein the virtual 3D object is displayed to a user on a 2D display device using the first scaling information; determining a second scaling based on the first scaling information; and displaying a 3D representation of the virtual 3D object on a 3D display device using the second scaling.

Description

[0001] This application is a divisional application of the invention patent application with application number 201980086103.5 filed on November 26, 2019 and invention name “Method for mirroring 3D objects onto a light field display”.

[0002] CROSS-REFERENCE TO RELATED APPLICATIONS

[0003] This application is a non-provisional application of U.S. Provisional Patent Application No. 62 / 774,062 (filed on November 30, 2018) entitled “METHOD FOR MIRRORING 3D OBJECTS TO LIGHTFIELDDISPLAYS” and claims the benefit of the U.S. Provisional Patent Application under 35 U.S.C. §119(e), which is incorporated herein by reference in its entirety. Background Art

[0004] Autostereoscopic displays such as light field displays can allow the human eye to see spatial information without the need for special glasses. Visualized light fields can be perceived within a given angle, called the field of view (FOV), measured from the screen direction. Horizontal-only parallax (HOP) light field displays or full parallax displays can provide angle-continuous visual effects horizontally and vertically. For example, a light field display can provide a 180-degree FOV in the horizontal direction and a 90-degree FOV in the vertical direction. Summary of the invention

[0005] Solutions that support the combined use of 2D displays and light field displays may be particularly useful. For example, a 2D display may be used in a mobile device, while a light field display may be used in a physical environment, such as a home, office, or vehicle. It may be useful to enable users of a 2D display to utilize the visualization capabilities of a light field display to present content that is typically available on a mobile device and a 2D display. It may be useful to enable content to be transferred to a light field display that can simultaneously provide separate views to viewers in different viewing directions. Transferring a 3D object from a 2D display to a light field display may require scaling and / or rotating the 3D object to achieve the desired size and orientation on the light field display. It may be particularly useful to consider the size difference between the 2D display and the light field display when scaling the size of the 3D object. In addition, it may be preferred to allow multiple users to mirror objects to the FOV of the light field display so that each mirrored 3D object is presented for the viewing direction of the corresponding user.

[0006] The present invention describes systems and methods for mirroring 3D objects to light field displays. Some embodiments include fast preview for mirrored objects in light field displays. In some embodiments, motion and gesture-based adjustment of 3D mirroring parameters is provided.

[0007] In some embodiments, the scaling and orientation of the mirrored content is based on the size and relative orientation of the displays.Some embodiments operate to create the effect of moving real objects by maintaining the size and orientation of 3D objects from a given viewing angle.

[0008] In some embodiments, the relative position of the source device and the light field display is used to control the positioning of the content as it moves from the source device to the light field display. In some embodiments, 3D mirroring the content from the source device to the light field display includes sending data identifying the 3D content and scaling parameters and orientation parameters without sending the 3D content itself.

[0009] Some embodiments are operable to enhance the use of light field displays and mobile devices and 2D displays. Some embodiments are operable to provide more realistic representations of digital 3D objects.

[0010] In some embodiments, a method for mirroring a 3D object from a first display to a second display is provided. The method includes determining the relative orientation and relative size of the first display and the second display. The method also includes selecting a 3D mirror mode. The method also includes calculating a set of viewing parameters based on the relative orientation, relative size and 3D mirror mode. The method also includes transmitting a description of the 3D object and the set of viewing parameters to the second display. In some embodiments, the set of viewing parameters includes an orientation parameter. In some embodiments, the set of viewing parameters includes a scaling parameter. In some embodiments, determining the relative orientation includes determining the relative position based on a SLAM model generated at the first display, wherein the SLAM model identifies the position of the second display. In some embodiments, determining the relative orientation includes using a smart space to determine the relative orientation. In some embodiments, the method also includes determining the position of the viewer relative to the first display and the second display, wherein calculating the set of viewing parameters is based on the position of the viewer. In some embodiments, the set of viewing parameters is calculated so that from the determined viewer's position, the apparent size of the 3D object on the first display is equal to the apparent size of the 3D object on the second display. In some embodiments, the set of viewing parameters is calculated such that, from the determined position of the viewer, the orientation of the 3D object on the first display is equal to the orientation of the 3D object on the second display.

[0011] In some embodiments, a method for mirroring a 3D object from a first display to a second display is provided. The method includes receiving a 3D content description from the first display. The method also includes receiving a set of viewing parameters from the first display. The method also includes receiving the 3D content based on the received 3D content description. The method also includes rendering a light field visualization at the second display based on the received 3D content and the set of viewing parameters. In some embodiments, the set of viewing parameters includes an orientation parameter. In some embodiments, the set of viewing parameters includes a scaling parameter. In some embodiments, the method also includes determining the relative orientation of the first display and the second display. In some embodiments, receiving the 3D content includes receiving the 3D content based on the relative orientation of the first display and the second display. In some embodiments, the method also includes tracking the relative position of the first display. In some such embodiments, rendering the light field visualization includes rendering the light field visualization based on the relative position of the first display. BRIEF DESCRIPTION OF THE DRAWINGS

[0012] Figure 1A is a system diagram illustrating an exemplary communication system in which one or more disclosed embodiments may be implemented.

[0013] Figure 1B is a diagram showing that according to the embodiment Figure 1A A system diagram of an exemplary wireless transmit / receive unit (WTRU) for use within a communication system is shown.

[0014] Figure 1C is a diagram showing that according to the embodiment, Figure 1A A system diagram of an exemplary radio access network (RAN) and an exemplary core network (CN) for use within a communication system is shown.

[0015] Figure 1D is a diagram showing that according to the embodiment Figure 1A A system diagram of another exemplary RAN and another exemplary CN for use within the communication system shown.

[0016] Figure 2 Depicted is an example of a light field display with 3D mirrored areas according to some embodiments.

[0017] Figure 3 An example of presenting instructions to a user for activating 3D mirroring in a 3D mirroring zone according to some embodiments is shown.

[0018] Figure 4 Depicted is an example preview view of a mirrored 3D object rendered in a light field display.

[0019] Figure 5Depicted is a 3D mirror zone presenting a full light field visualization of a mirrored 3D object according to some embodiments.

[0020] Figure 6 is a schematic diagram of an environment for mirroring 3D objects from a mobile device to a light field display according to some embodiments.

[0021] Fig. 7A An overview of a process for mirroring a 3D object to a light field display is depicted according to some embodiments.

[0022] Figure 7B Depicted are exemplary sub-processes for activating mirroring of a 3D object according to some embodiments.

[0023] Figure 7C Depicted are exemplary sub-processes for preparing a mirror image of a 3D object according to some embodiments.

[0024] Fig.7D Depicted are exemplary sub-processes for outputting a mirrored 3D object according to some embodiments.

[0025] Fig. 8A and 8B is a sequence diagram of a process for mirroring a 3D object into a field of view of a light field display according to some embodiments.

[0026] Fig. 9 An exemplary scenario is shown in which a user utilizes proximity and device position based triggering of 3D mirroring to transfer a 3D object from a mobile device to a light field display in accordance with some embodiments.

[0027] Fig.10 Depicted is an example of mirroring a 3D object from a personal display to a group display while maintaining the orientation and scaling of the 3D object from the viewer's perspective, according to some embodiments.

[0028] Fig.11 is a sequence diagram of a first exemplary process for mirroring 3D content from an originating device to a light field display according to some embodiments.

[0029] Fig.12 is a sequence diagram of a second exemplary process for mirroring 3D content from an originating device to a light field display according to some embodiments.

[0030] FIG. 13A to FIG. 13D are schematic front views (lower row) and top views (upper row) of a 2D display device showing different object orientations for display of virtual objects.

[0031] Fig.14Ais a schematic top view illustrating the orientation of virtual objects and their representations in a method of mirroring 3D content in some embodiments.

[0032] Fig. 14B is based on Fig.14A A schematic diagram of an embodiment showing a view of a 3D display from a user's perspective.

[0033] Fig.15A is a schematic top view illustrating the orientation of virtual objects and their representations in a method of mirroring 3D content in some embodiments.

[0034] Fig. 15B is based on Fig.15A A schematic diagram of an embodiment showing a view of a 3D display from a user's perspective.

[0035] Fig.16 Depicted is an example scenario for calculating a zoom ratio based on the size of two displays and the distance from each display to a viewer, according to some embodiments.

[0036] Example Network for Implementing Embodiments

[0037] Figure 1A 1 is a schematic diagram illustrating an exemplary communication system 100 in which one or more disclosed embodiments may be implemented. The communication system 100 may be a multiple access system that provides content such as voice, data, video, messaging, broadcast, etc. to multiple wireless users. The communication system 100 may enable multiple wireless users to access such content through sharing of system resources (including wireless bandwidth). For example, the communication system 100 may employ one or more channel access methods such as code division multiple access (CDMA), time division multiple access (TDMA), frequency division multiple access (FDMA), orthogonal FDMA (OFDMA), single carrier FDMA (SC-FDMA), zero tail unique word DFT spread OFDM (ZT UW DTS-s OFDM), unique word OFDM (UW-OFDM), resource block filtered OFDM, filter bank multi-carrier (FBMC), etc.

[0038] like Figure 1AAs shown, the communication system 100 may include wireless transmit / receive units (WTRUs) 102a, 102b, 102c, 102d, RAN 104 / 113, CN 106 / 115, public switched telephone network (PSTN) 108, Internet 110 and other networks 112, but it should be understood that the disclosed embodiments contemplate any number of WTRUs, base stations, networks, and / or network elements. Each of the WTRUs 102a, 102b, 102c, 102d may be any type of device configured to operate and / or communicate in a wireless environment. By way of example, the WTRUs 102a, 102b, 102c, 102d (any of which may be referred to as a “station” and / or “STA”) may be configured to transmit and / or receive wireless signals and may include user equipment (UE), a mobile station, a fixed or mobile subscriber unit, a subscription-based unit, a pager, a cellular phone, a personal digital assistant (PDA), a smart phone, a laptop, a netbook, a personal computer, a wireless sensor, a hotspot or Mi-Fi device, an Internet of Things (IoT) device, a watch or other wearable device, a head-mounted display (HMD), a vehicle, a drone, medical devices and applications (e.g., remote surgery), industrial devices and applications (e.g., robots and / or other wireless devices operating in an industrial and / or automated process chain environment), a consumer electronic device, a device operating on a commercial and / or industrial wireless network, etc. Any of the WTRUs 102a, 102b, 102c, and 102d may be interchangeably referred to as a UE.

[0039] The communication system 100 may also include a base station 114a and / or a base station 114b. Each of the base stations 114a, 114b may be any type of device that is configured to wirelessly interface with at least one of the WTRUs 102a, 102b, 102c, 102d to facilitate access to one or more communication networks, such as the CN 106 / 115, the Internet 110, and / or other networks 112. By way of example, the base stations 114a, 114b may be a base transceiver station (BTS), a Node B, an eNode B, a Home Node B, a Home eNode B, a gNB, an NR Node B, a site controller, an access point (AP), a wireless router, and the like. While the base stations 114a, 114b are each depicted as a single element, it will be appreciated that the base stations 114a, 114b may include any number of interconnected base stations and / or network elements.

[0040] The base station 114a may be part of the RAN 104 / 113, which may also include other base stations and / or network elements (not shown), such as a base station controller (BSC), a radio network controller (RNC), a relay node, etc. The base station 114a and / or the base station 114b may be configured to transmit and / or receive wireless signals on one or more carrier frequencies (which may be referred to as cells (not shown)). These frequencies may be in a licensed spectrum, an unlicensed spectrum, or a combination of licensed and unlicensed spectrum. A cell may provide coverage of wireless services to a specific geographic area, which may be relatively fixed or may change over time. The cell may be further divided into cell sectors. For example, a cell associated with the base station 114a may be divided into three sectors. Therefore, in one embodiment, the base station 114a may include three transceivers, i.e., one transceiver for each sector of the cell. In one embodiment, the base station 114a may employ multiple-input multiple-output (MIMO) technology and may use multiple transceivers for each sector of the cell. For example, beamforming may be used to transmit and / or receive signals in desired spatial directions.

[0041] The base stations 114a, 114b may communicate with one or more of the WTRUs 102a, 102b, 102c, 102d over an air interface 116, which may be any suitable wireless communication link (e.g., radio frequency (RF), microwave, centimeter wave, micrometer wave, infrared (IR), ultraviolet (UV), visible light, etc.). The air interface 116 may be established using any suitable radio access technology (RAT).

[0042] More specifically, as noted above, the communication system 100 may be a multiple access system and may employ one or more channel access schemes such as CDMA, TDMA, FDMA, OFDMA, SC-FDMA, etc. For example, the base station 114a in the RAN 104 / 113 and the WTRUs 102a, 102b, 102c may implement a radio technology such as Universal Mobile Telecommunications System (UMTS) Terrestrial Radio Access (UTRA), which may use Wideband CDMA (WCDMA) to establish the air interface 115 / 116 / 117. WCDMA may include communication protocols such as High Speed ​​Packet Access (HSPA) and / or Evolved HSPA (HSPA+). HSPA may include High Speed ​​Downlink (DL) Packet Access (HSDPA) and / or High Speed ​​UL Packet Access (HSUPA).

[0043] In one embodiment, the base station 114a and the WTRUs 102a, 102b, 102c may implement a radio technology such as Evolved UMTS Terrestrial Radio Access (E-UTRA), which may establish the air interface 116 using Long Term Evolution (LTE) and / or Advanced LTE (LTE-A) and / or Advanced LTE Pro (LTE-A Pro).

[0044] In one embodiment, the base station 114a and the WTRUs 102a, 102b, 102c may implement a radio technology such as NR radio access, which may establish the air interface 116 using New Radio (NR).

[0045] In one embodiment, the base station 114a and the WTRUs 102a, 102b, 102c may implement multiple radio access technologies. For example, the base station 114a and the WTRUs 102a, 102b, 102c may implement LTE radio access and NR radio access together, for example using dual connectivity (DC) principles. Thus, the air interface utilized by the WTRUs 102a, 102b, 102c may be characterized by multiple types of radio access technologies and / or transmissions sent to / from multiple types of base stations (e.g., eNBs and gNBs).

[0046] In other embodiments, the base station 114a and the WTRUs 102a, 102b, 102c may implement radio technologies such as IEEE 802.11 (i.e., Wireless Fidelity (WiFi)), IEEE 802.16 (i.e., Worldwide Interoperability for Microwave Access (WiMAX)), CDMA2000, CDMA2000 1X, CDMA2000 EV-DO, Interim Standard 2000 (IS-2000), Interim Standard 95 (IS-95), Interim Standard 856 (IS-856), Global System for Mobile Communications (GSM), GSM Enhanced Data Rates for Evolution (EDGE), GSM EDGE (GERAN), etc.

[0047] Figure 1AThe base station 114b in the may be, for example, a wireless router, a Home NodeB, a Home eNodeB, or an access point, and may utilize any suitable RAT to facilitate wireless connectivity in a local area, such as a business location, a home, a vehicle, a campus, an industrial facility, an air corridor (e.g., for use by drones), a road, and the like. In one embodiment, the base station 114b and the WTRUs 102c, 102d may implement a radio technology such as IEEE 802.11 to establish a wireless local area network (WLAN). In one embodiment, the base station 114b and the WTRUs 102c, 102d may implement a radio technology such as IEEE 802.15 to establish a wireless personal area network (WPAN). In yet another embodiment, the base station 114b and the WTRUs 102c, 102d may utilize a cellular-based RAT (e.g., WCDMA, CDMA2000, GSM, LTE, LTE-A, LTE-A Pro, NR, etc.) to establish a picocell or a femtocell. As Figure 1A As shown, base station 114b may have a direct connection to Internet 110. Therefore, base station 114b may not need to access Internet 110 via CN 106 / 115.

[0048] The RAN 104 / 113 may be in communication with the CN 106 / 115, which may be any type of network configured to provide voice, data, applications and / or Voice over Internet Protocol (VoIP) services to one or more of the WTRUs 102a, 102b, 102c, 102d. The data may have different quality of service (QoS) requirements, such as different throughput requirements, delay requirements, error tolerance requirements, reliability requirements, data throughput requirements, mobility requirements, etc. The CN 106 / 115 may provide call control, billing services, mobile location-based services, pre-paid calling, Internet connectivity, video distribution, etc., and / or perform high-level security functions, such as user authentication. Although not described in detail in the specification, the CN 106 / 115 may be configured to provide voice, data, applications and / or Voice over Internet Protocol (VoIP) services to one or more of the WTRUs 102a, 102b, 102c, 102d. The data may have different quality of service (QoS) requirements, such as different throughput requirements, delay requirements, error tolerance requirements, reliability requirements, data throughput requirements, mobility requirements, etc. Figure 1A Although not shown in the figure, it will be appreciated that the RAN 104 / 113 and / or the CN 106 / 115 may be in direct or indirect communication with other RANs that employ the same RAT as the RAN 104 / 113 or a different RAT. For example, in addition to being connected to the RAN 104 / 113, which may utilize NR radio technology, the CN 106 / 115 may also be in communication with another RAN (not shown) that employs GSM, UMTS, CDMA 2000, WiMAX, E-UTRA, or WiFi radio technology.

[0049] The CN 106 / 115 may also serve as a gateway for the WTRUs 102a, 102b, 102c, 102d to access the PSTN 108, the Internet 110, and / or other networks 112. The PSTN 108 may include a circuit-switched telephone network that provides plain old telephone service (POTS). The Internet 110 may include a global system of interconnected computer networks and devices that use common communication protocols such as the Transmission Control Protocol (TCP), the User Datagram Protocol (UDP), and / or the Internet Protocol (IP) in the TCP / IP Internet protocol suite. The networks 112 may include wired and / or wireless communication networks owned and / or operated by other service providers. For example, the networks 112 may include another CN connected to one or more RANs, which may employ the same RAT as the RAN 104 / 113 or a different RAT.

[0050] Some or all of the WTRUs 102a, 102b, 102c, 102d in the communication system 100 may include multi-mode capabilities (e.g., the WTRUs 102a, 102b, 102c, 102d may include multiple transceivers for communicating with different wireless networks via different wireless links). Figure 1A The illustrated WTRU 102c may be configured to communicate with the base station 114a, which may employ a cellular-based radio technology, and with the base station 114b, which may employ an IEEE 802 radio technology.

[0051] Figure 1B is a system diagram illustrating an exemplary WTRU 102. Figure 1B As shown, the WTRU 102 may include, among other things, a processor 118, a transceiver 120, a transmit / receive element 122, a speaker / microphone 124, a keypad 126, a display / touchpad 128, non-removable memory 130, removable memory 132, a power source 134, a global positioning system (GPS) chipset 136, and / or other peripherals 138. It will be appreciated that the WTRU 102 may include any subcombination of the foregoing elements while remaining consistent with an embodiment.

[0052] The processor 118 may be a general purpose processor, a special purpose processor, a conventional processor, a digital signal processor (DSP), a plurality of microprocessors, one or more microprocessors associated with a DSP core, a controller, a microcontroller, an application specific integrated circuit (ASIC), a field programmable gate array (FPGA) circuit, any other type of integrated circuit (IC), a state machine, etc. The processor 118 may perform signal coding, data processing, power control, input / output processing, and / or any other functionality that enables the WTRU 102 to operate in a wireless environment. The processor 118 may be coupled to the transceiver 120, which may be coupled to the transmit / receive element 122. Although Figure 1B The processor 118 and the transceiver 120 are depicted as separate components, but it is understood that the processor 118 and the transceiver 120 may be integrated together in an electronic package or chip.

[0053] The transmit / receive element 122 may be configured to transmit signals to or receive signals from a base station (e.g., base station 114a) via an air interface 116. For example, in one embodiment, the transmit / receive element 122 may be an antenna configured to transmit and / or receive RF signals. For example, in one embodiment, the transmit / receive element 122 may be an emitter / detector configured to transmit and / or receive IR, UV, or visible light signals. In yet another embodiment, the transmit / receive element 122 may be configured to transmit and / or receive RF and light signals. It should be appreciated that the transmit / receive element 122 may be configured to transmit and / or receive any combination of wireless signals.

[0054] Although the transmit / receive element 122 is Figure 1B 1 as a single element, the WTRU 102 may include any number of transmit / receive elements 122. More specifically, the WTRU 102 may employ MIMO technology. Thus, in one embodiment, the WTRU 102 may include two or more transmit / receive elements 122 (e.g., multiple antennas) for transmitting and receiving wireless signals over the air interface 116.

[0055] The transceiver 120 may be configured to modulate signals to be transmitted by the transmit / receive element 122 and to demodulate signals received by the transmit / receive element 122. As noted above, the WTRU 102 may have multi-mode capabilities. Thus, the transceiver 120 may include multiple transceivers to enable the WTRU 102 to communicate via multiple RATs, such as NR and IEEE 802.11.

[0056] The processor 118 of the WTRU 102 may be coupled to a speaker / microphone 124, a keypad 126, and / or a display / touchpad 128 (e.g., a liquid crystal display (LCD) display unit or an organic light emitting diode (OLED) display unit) and may receive user input data therefrom. The processor 118 may also output user data to the speaker / microphone 124, the keypad 126, and / or the display / touchpad 128. In addition, the processor 118 may access information from and store data in any type of suitable memory, such as a non-removable memory 130 and / or a removable memory 132. The non-removable memory 130 may include a random access memory (RAM), a read-only memory (ROM), a hard disk, or any other type of memory storage device. The removable memory 132 may include a subscriber identity module (SIM) card, a memory stick, a secure digital (SD) memory card, and the like. In other embodiments, the processor 118 may access information from and store data in a memory that is not physically located on the WTRU 102, such as on a server or a home computer (not shown).

[0057] The processor 118 may receive power from the power source 134 and may be configured to distribute power to and / or control the power to other components in the WTRU 102. The power source 134 may be any suitable device for powering the WTRU 102. For example, the power source 134 may include one or more dry cell batteries (e.g., nickel-cadmium (NiCd), nickel-zinc (NiZn), nickel-metal hydride (NiMH), lithium-ion (Li-ion), etc.), solar cells, fuel cells, etc.

[0058] The processor 118 may also be coupled to the GPS chipset 136, which may be configured to provide location information (e.g., longitude and latitude) regarding the current location of the WTRU 102. In addition to or in lieu of the information from the GPS chipset 136, the WTRU 102 may also receive location information from a base station (e.g., base stations 114a, 114b) over the air interface 116 and / or determine its location based on the time at which signals are received from two or more nearby base stations. It will be appreciated that the WTRU 102 may acquire location information by any suitable location-determination method while remaining consistent with an embodiment.

[0059] The processor 118 may also be coupled to other peripherals 138, which may include one or more software modules and / or hardware modules that provide additional features, functionality, and / or wired or wireless connectivity. For example, the peripherals 138 may include an accelerometer, an electronic compass, a satellite transceiver, a digital camera (for photos and / or video), a universal serial bus (USB) port, a vibration device, a television transceiver, a hands-free headset, module, a frequency modulation (FM) radio unit, a digital music player, a media player, a video game player module, an Internet browser, a virtual reality and / or augmented reality (VR / AR) device, an activity tracker, etc. Peripheral device 138 may include one or more sensors, which may be one or more of the following: a gyroscope, an accelerometer, a Hall effect sensor, a magnetometer, an orientation sensor, a proximity sensor, a temperature sensor, a time sensor; a geolocation sensor; an altimeter, a light sensor, a touch sensor, a magnetometer, a barometer, a gesture sensor, a biometric sensor, and / or a humidity sensor.

[0060] The WTRU 102 may include a full-duplex radio for which transmission and reception of some or all signals (e.g., associated with specific subframes for UL (e.g., for transmission) and downlink (e.g., for reception)) may be concurrent and / or simultaneous. The full-duplex radio may include an interference management unit to reduce and / or substantially eliminate self-interference via hardware (e.g., choke) or via signal processing by a processor (e.g., a separate processor (not shown) or via the processor 118). In one embodiment, the WTRU 102 may include a half-duplex radio for which transmission and reception of some or all signals (e.g., associated with specific subframes for UL (e.g., for transmission) or downlink (e.g., for reception)) may be concurrent and / or simultaneous.

[0061] Figure 1C 1 is a system diagram illustrating the RAN 104 and the CN 106 according to one embodiment. As described above, the RAN 104 may employ an E-UTRA radio technology to communicate with the WTRUs 102a, 102b, 102c over the air interface 116. The RAN 104 may also be in communication with the CN 106.

[0062] The RAN 104 may include evolved Node-Bs 160a, 160b, 160c, though it will be appreciated that the RAN 104 may include any number of evolved Node-Bs while remaining consistent with an embodiment. The evolved Node-Bs 160a, 160b, 160c may each include one or more transceivers for communicating with the WTRUs 102a, 102b, 102c over the air interface 116. In one embodiment, the evolved Node-Bs 160a, 160b, 160c may implement MIMO technology. Thus, the evolved Node-B 160a, for example, may use multiple antennas to transmit wireless signals to and / or receive wireless signals from the WTRU 102a.

[0063] Each of the evolved Node Bs 160a, 160b, 160c may be associated with a particular cell (not shown) and may be configured to handle radio resource management decisions, handover decisions, scheduling of users in the UL and / or DL, etc. Figure 1C As shown, the eNode-Bs 160a, 160b, 160c may communicate with one another via an X2 interface.

[0064] Figure 1C The illustrated CN 106 may include a mobility management entity (MME) 162, a serving gateway (SGW) 164, and a packet data network (PDN) gateway (or PGW) 166. While each of the foregoing elements is depicted as part of the CN 106, it should be understood that any of these elements may be owned and / or operated by an entity other than the CN operator.

[0065] The MME 162 may be connected to each of the evolved Node-Bs 162a, 162b, 162c in the RAN 104 via an S1 interface and may serve as a control node. For example, the MME 162 may be responsible for authenticating users of the WTRUs 102a, 102b, 102c, bearer activation / deactivation, selecting a particular serving gateway during an initial attach of the WTRUs 102a, 102b, 102c, etc. The MME 162 may provide a control plane function for switching between the RAN 104 and other RANs (not shown) that employ other radio technologies, such as GSM and / or WCDMA.

[0066] The SGW 164 may be connected to each of the evolved Node-Bs 160a, 160b, 160c in the RAN 104 via an S1 interface. The SGW 164 may generally route and forward user data packets to / from the WTRUs 102a, 102b, 102c. The SGW 164 may perform other functions, such as anchoring the user plane during inter-evolved Node-B handovers, triggering paging when DL data is available for the WTRUs 102a, 102b, 102c, managing and storing the context of the WTRUs 102a, 102b, 102c, and the like.

[0067] The SGW 164 may be connected to the PGW 166, which may provide the WTRUs 102a, 102b, 102c with access to packet-switched networks, such as the Internet 110, to facilitate communications between the WTRUs 102a, 102b, 102c and IP-enabled devices.

[0068] The CN 106 may facilitate communications with other networks. For example, the CN 106 may provide the WTRUs 102a, 102b, 102c with access to circuit-switched networks, such as the PSTN 108, to facilitate communications between the WTRUs 102a, 102b, 102c and traditional land-line communications devices. For example, the CN 106 may include, or may be in communication with, an IP gateway (e.g., an IP Multimedia Subsystem (IMS) server) that serves as an interface between the CN 106 and the PSTN 108. In addition, the CN 106 may provide the WTRUs 102a, 102b, 102c with access to other networks 112, which may include other wired and / or wireless networks owned and / or operated by other service providers.

[0069] Although the WTRU Figures 1A to 1D Although described as a wireless terminal, it is contemplated that in certain representative embodiments such a terminal may (eg, temporarily or permanently) employ a wired communications interface with a communications network.

[0070] In a representative embodiment, the other network 112 may be a WLAN.

[0071] A WLAN in infrastructure basic service set (BSS) mode may have an access point (AP) for a BSS and one or more stations (STAs) associated with the AP. The AP may have access or an interface to a distribution system (DS) or another type of wired / wireless network that carries traffic to and / or carries traffic away from the BSS. Traffic originating from outside the BSS and leading to the STA may be reached by the AP and may be delivered to the STA. Traffic originating from the STA and leading to a destination outside the BSS may be sent to the AP to be delivered to the corresponding destination. Traffic between STAs within the BSS may be sent by the AP, for example, wherein the source STA may send traffic to the AP, and the AP may deliver the traffic to the destination STA. Traffic between STAs within the BSS may be considered and / or referred to as point-to-point traffic. Point-to-point traffic may be sent between the source and destination STAs (e.g., directly between them) using a direct link setup (DLS). In certain representative embodiments, the DLS may use 802.11e DLS or 802.11z tunnel DLS (TDLS). A WLAN using an independent BSS (IBSS) mode may not have an AP, and STAs (eg, all STAs) within or using the IBSS may communicate directly with each other. The IBSS communication mode may sometimes be referred to herein as an "ad-hoc" communication mode.

[0072] When using the 802.11ac infrastructure operating mode or a similar operating mode, the AP may transmit a beacon on a fixed channel (such as a primary channel). The primary channel may be a fixed width (e.g., a 20 MHz wide bandwidth) or a width dynamically set via signaling. The primary channel may be an operating channel of the BSS and may be used by the STA to establish a connection with the AP. In certain representative embodiments, carrier sense multiple access / collision avoidance (CSMA / CA) may be implemented, for example, in an 802.11 system. For CSMA / CA, a STA (e.g., each STA) (including the AP) may sense the primary channel. If the primary channel is sensed / detected and / or determined to be busy by a particular STA, the particular STA may back off. One STA (e.g., only one station) may transmit at any given time in a given BSS.

[0073] High throughput (HT) STAs may communicate using a 40 MHz wide channel, for example, via a combination of a primary 20 MHz channel and adjacent or non-adjacent 20 MHz channels to form a 40 MHz wide channel.

[0074] Very high throughput (VHT) STA can support 20MHz, 40MHz, 80MHz and / or 160MHz wide channels. 40MHz and / or 80MHz channels can be formed by combining continuous 20MHz channels. A 160MHz channel can be formed by combining 8 continuous 20MHz channels, or by combining two non-continuous 80MHz channels (this can be called 80+80 configuration). For the 80+80 configuration, after channel coding, the data can pass through a segment parser that can divide the data into two streams. Each stream can be processed by inverse fast Fourier transform (IFFT) and time domain processing separately. These streams can be mapped to two 80MHz channels, and data can be transmitted by transmitting STA. At the receiver of the receiving STA, the above-mentioned operation for the 80+80 configuration can be reversed, and the combined data can be sent to the medium access control (MAC).

[0075] 802.11af and 802.11ah support operating modes below 1GHz. Channel operating bandwidths and carriers are reduced in 802.11af and 802.11ah relative to those used in 802.11n and 802.11ac. 802.11af supports 5MHz, 10MHz, and 20MHz bandwidths in the TV White Space (TVWS) spectrum, and 802.11ah supports 1MHz, 2MHz, 4MHz, 8MHz, and 16MHz bandwidths using non-TVWS spectrum. According to a representative embodiment, 802.11ah may support instrument type control / machine type communications, such as MTC devices in macro coverage areas. MTC devices may have certain capabilities, such as limited capabilities, including support for (e.g., only support for) certain bandwidths and / or limited bandwidths. MTC devices may include batteries with battery life above a threshold (e.g., to maintain very long battery life).

[0076] WLAN systems that can support multiple channels and channel bandwidths such as 802.11n, 802.11ac, 802.11af, and 802.11ah include channels that can be designated as primary channels. The primary channel may have a bandwidth equal to the maximum common operating bandwidth supported by all STAs in the BSS. The bandwidth of the primary channel may be set and / or limited by a STA (which supports the minimum bandwidth operating mode) from all STAs operating in the BSS. In the example of 802.11ah, for STAs (e.g., MTC-type devices) that support (e.g., only support) 1MHz mode, the primary channel may be 1MHz wide, even if the AP and other STAs in the BSS support 2MHz, 4MHz, 8MHz, 16MHz, and / or other channel bandwidth operating modes. Carrier sensing and / or network allocation vector (NAV) settings may depend on the state of the primary channel. If the primary channel is busy, for example, because a STA (supporting only the 1MHz operating mode) is transmitting to the AP, the entire available frequency band may be considered busy even if most of the frequency bands remain idle and may be available.

[0077] In the United States, the available frequency band for 802.11ah is 902MHz to 928MHz. In South Korea, the available frequency band is 917.5MHz to 923.5MHz. In Japan, the available frequency band is 916.5MHz to 927.5MHz. The total bandwidth available for 802.11ah is 6MHz to 26MHz, depending on the country code.

[0078] Figure 1D 1 is a system diagram showing the RAN 113 and the CN 115 according to one embodiment. As noted above, the RAN 113 may employ NR radio technology to communicate with the WTRUs 102a, 102b, 102c over the air interface 116. The RAN 113 may also be in communication with the CN 115.

[0079] The RAN 113 may include gNBs 180a, 180b, 180c, though it will be appreciated that the RAN 113 may include any number of gNBs while remaining consistent with an embodiment. The gNBs 180a, 180b, 180c may each include one or more transceivers to communicate with the WTRUs 102a, 102b, 102c over the air interface 116. In one embodiment, the gNBs 180a, 180b, 180c may implement MIMO technology. For example, the gNBs 180a, 180b may utilize beamforming to transmit signals to and / or receive signals from the gNBs 180a, 180b, 180c. Thus, the gNB 180a may, for example, use multiple antennas to transmit wireless signals to and / or receive wireless signals from the WTRU 102a. In one embodiment, the gNBs 180a, 180b, 180c may implement carrier aggregation techniques. For example, the gNB 180a may transmit multiple component carriers to the WTRU 102a (not shown). A subset of these component carriers may be on unlicensed spectrum, while the remaining component carriers may be on licensed spectrum. In one embodiment, the gNBs 180a, 180b, 180c may implement coordinated multi-point (CoMP) techniques. For example, the WTRU 102a may receive coordinated transmissions from gNB 180a and gNB 180b (and / or gNB 180c).

[0080] The WTRUs 102a, 102b, 102c may communicate with the gNBs 180a, 180b, 180c using transmissions associated with scalable parameter sets. For example, OFDM symbol spacing and / or OFDM subcarrier spacing may vary for different transmissions, different cells, and / or different portions of the wireless transmission spectrum. The WTRUs 102a, 102b, 102c may communicate with the gNBs 180a, 180b, 180c using subframes or transmission time intervals (TTIs) of varying or scalable lengths (e.g., containing different numbers of OFDM symbols and / or continuously varying absolute time lengths).

[0081] The gNBs 180a, 180b, 180c may be configured to communicate with the WTRUs 102a, 102b, 102c in a standalone configuration and / or a non-standalone configuration. In a standalone configuration, the WTRUs 102a, 102b, 102c may communicate with the gNBs 180a, 180b, 180c while also not accessing other RANs (e.g., such as the eNodeBs 160a, 160b, 160c). In a standalone configuration, the WTRUs 102a, 102b, 102c may use one or more of the gNBs 180a, 180b, 180c as mobility anchor points. In a standalone configuration, the WTRUs 102a, 102b, 102c may communicate with the gNBs 180a, 180b, 180c using signals in an unlicensed band. In a non-standalone configuration, the WTRUs 102a, 102b, 102c may communicate or connect with the gNBs 180a, 180b, 180c while also communicating or connecting with other RANs, such as the eNode-Bs 160a, 160b, 160c. For example, the WTRUs 102a, 102b, 102c may implement the DC principle to communicate with one or more gNBs 180a, 180b, 180c and one or more evolved Node-Bs 160a, 160b, 160c substantially simultaneously. In a non-standalone configuration, the evolved Node-Bs 160a, 160b, 160c may serve as mobility anchors for the WTRUs 102a, 102b, 102c, and the gNBs 180a, 180b, 180c may provide additional coverage and / or throughput for serving the WTRUs 102a, 102b, 102c.

[0082] Each of the gNBs 180a, 180b, 180c may be associated with a particular cell (not shown) and may be configured to handle radio resource management decisions, handover decisions, scheduling of users in UL and / or DL, support of network slicing, dual connectivity, interworking between NR and E-UTRA, routing of user plane data towards a user plane function (UPF) 184a, 184b, routing of control plane information towards an access and mobility management function (AMF) 182a, 182b, etc. Figure 1D As shown, gNBs 180a, 180b, and 180c may communicate with each other via an Xn interface.

[0083] Figure 1DThe illustrated CN 115 may include at least one AMF 182a, 182b, at least one UPF 184a, 184b, at least one session management function (SMF) 183a, 183b, and possible data networks (DNs) 185a, 185b. Although each of the foregoing elements is depicted as part of the CN 115, it should be understood that any of these elements may be owned and / or operated by an entity other than the CN operator.

[0084] The AMF 182a, 182b may be connected to one or more of the gNBs 180a, 180b, 180c via the N2 interface in the RAN 113 and may serve as a control node. For example, the AMF 182a, 182b may be responsible for authenticating users of the WTRU 102a, 102b, 102c, support of network slicing (e.g., handling of different PDU sessions with different requirements), selecting a specific SMF 183a, 183b, management of registration areas, termination of NAS signaling, mobility management, etc. The AMF 182a, 182b may use network slicing in order to customize CN support for the WTRU 102a, 102b, 102c based on the type of services used by the WTRU 102a, 102b, 102c. For example, different network slices may be established for different use cases, such as services relying on ultra-reliable low latency (URLLC) access, services relying on enhanced mobile broadband (eMBB) access, services for machine type communication (MTC) access, etc. The AMF 162 may provide a control plane function for switching between the RAN 113 and other RANs (not shown) that employ other radio technologies, such as LTE, LTE-A, LTE-A Pro, and / or non-3GPP access technologies, such as WiFi.

[0085] The SMF 183a, 183b may be connected to the AMF 182a, 182b in the CN 115 via the N11 interface. The SMF 183a, 183b may also be connected to the UPF 184a, 184b in the CN 115 via the N4 interface. The SMF 183a, 183b may select and control the UPF 184a, 184b, and configure traffic routing through the UPF 184a, 184b. The SMF 183a, 183b may perform other functions such as managing and allocating UE IP addresses, managing PDU sessions, controlling policy enforcement and QoS, providing downlink data notification, etc. The PDU session type may be IP-based, non-IP-based, Ethernet-based, etc.

[0086] The UPF 184a, 184b may be connected to one or more of the gNBs 180a, 180b, 180c in the RAN 113 via the N3 interface, which may provide the WTRUs 102a, 102b, 102c with access to packet-switched networks, such as the Internet 110, to facilitate communications between the WTRUs 102a, 102b, 102c and IP-enabled devices. The UPF 184, 184b may perform other functions, such as routing and forwarding packets, enforcing user plane policies, supporting multi-homed PDU sessions, handling user plane QoS, buffering downlink packets, providing mobility anchoring, etc.

[0087] The CN 115 may facilitate communications with other networks. For example, the CN 115 may include or may communicate with an IP gateway (e.g., an IP Multimedia Subsystem (IMS) server) that serves as an interface between the CN 115 and the PSTN 108. In addition, the CN 115 may provide the WTRUs 102a, 102b, 102c with access to other networks 112, which may include other wired and / or wireless networks owned and / or operated by other service providers. In one embodiment, the WTRUs 102a, 102b, 102c may be connected to local data networks (DNs) 185a, 185b via the UPF 184a, 184b via an N3 interface to the UPF 184a, 184b and an N6 interface between the UPF 184a, 184b and the DN 185a, 185b.

[0088] Given that Figures 1A to 1D as well as Figures 1A to 1D Corresponding to the description of the present invention, one or more or all of the functions described herein with reference to one or more of the following may be performed by one or more simulation devices (not shown): WTRU102a-d, base station 114a-b, evolved Node B 160a-c, MME 162, SGW 164, PGW 166, gNB 180a-c, AMF182a-b, UPF 184a-b, SMF 183a-b, DN 185a-b and / or any other device described herein. The simulation device may be one or more devices configured to emulate one or more or all of the functions described herein. For example, the simulation device may be used to test other devices and / or simulate network and / or WTRU functions.

[0089] The simulation device can be designed to implement one or more tests of other devices in a laboratory environment and / or an operator network environment. For example, the one or more simulation devices can perform one or more or all functions while being fully or partially implemented and / or deployed as part of a wired and / or wireless communication network to test other devices within the communication network. The one or more simulation devices can perform the one or more functions or all functions while being temporarily implemented / deployed as part of a wired and / or wireless communication network. The simulation device can be directly coupled to another device for testing purposes and / or can use over-the-air wireless communications to perform testing.

[0090] The one or more simulation devices may perform one or more (including all) functions without being implemented / deployed as part of a wired and / or wireless communication network. For example, the simulation device may be used in a test scenario in a test lab and / or a non-deployed (e.g., testing) wired and / or wireless communication network to implement testing of one or more components. The one or more simulation devices may be test equipment. Direct RF coupling and / or wireless communication via RF circuits (e.g., which may include one or more antennas) may be used by the simulation device to transmit and / or receive data. DETAILED DESCRIPTION

[0091] In some embodiments, the method of 3D mirroring enables setting the initial size, orientation, and position of a 3D object mirrored to a light field display. In some embodiments, the main viewing direction or "object normal" of the 3D object is determined. The object normal may be, for example, a direction perpendicular to the main surface of the object. The 3D object can be transformed by comparing the surface of a 2D display displaying the 3D object to be mirrored with the object normal. The mirrored 3D content may be configured so that the normal of the 3D content is oriented toward the position of the viewer relative to the light field display. The size of the mirrored 3D content may depend on, for example, its relative size in the 2D display. For example, 3D content presented in full screen mode in a 2D display may be mirrored in a light field display and presented in full screen mode. In some embodiments, the size of the 3D content is scaled based on the distance between the viewer and the center point of the light field display. For example, for longer viewing distances, the 3D content may be scaled larger. In some embodiments, the depth of the 3D object in the light field display may be initially adjusted so that the clipping of the 3D object in different viewing directions is minimized.

[0092] In some embodiments, the FOV of the light field display and the physical environment can be segmented. For example, the FOV of the light field display can be divided to provide separate views, such as for different users, for various 3D objects, and / or for presenting views of 3D objects in different orientations (e.g., front view and side view).

[0093] Some embodiments operate to enable adjustment of parameters for mirroring a 3D object to a light field display. For example, a user may adjust the size and orientation of a mirrored 3D object in a light field display.

[0094] Some embodiments operate to enable fast switching to 3D mirror mode. For example, a fast preview light field visualization of a 3D object may be provided first, and a full quality light field visualization may be provided later.

[0095] 1. Overview of some 3D mirror implementation schemes

[0096] 1.1 3D Mirror Area

[0097] The FOV of the light field display can be shared for a desired number of 3D mirror zones. Each mirror zone can present a mirrored 3D object to a corresponding user. In some embodiments, the mirror zone can provide guidance to the user to assist in 3D mirroring. For example, Figure 2 A light field display 202 is depicted that provides a 180 degree field of view (FOV) shared by three 3D mirror zones in the horizontal direction. Each mirror zone can be used to mirror a 3D object to the light field display (e.g., in a mirror image of a car model). The 3D mirror zones can be implemented in the light field display, for example, as an array of angular views. The 3D objects that appear in the 3D mirror zones may only be visible to viewers in the viewing direction of the 3D mirror zones. Therefore, the 3D mirror zones can be used not only as areas where content for a specific user is mirrored in the 3D display, but also as a privacy function, where the mirrored content can be limited to viewing at a specific FOV. For example, the first virtual object 204 can be visible from within the zone 206, the second virtual object 208 can be visible from within the zone 201, and the third virtual object 212 can be visible from within the zone 214.

[0098] Various 3D mirroring actions may be useful in different situations of mirroring content to a light field display. As a first example, limited 3D mirroring actions may be implemented such that mirroring content to one 3D object mirroring zone results in display of the content visible only to viewers in the same zone. As a second example, broadcast 3D mirroring actions may be implemented such that a person in one 3D object mirroring zone may make the content visible to all viewers of the light field display (e.g., by displaying the content to multiple or all 3D mirroring zones of the light field display).

[0099] In some embodiments, a 3D mirror zone can be used to present instructions for activating a 3D mirror in a light field display. The 3D mirror zone can provide instructions to the user before the 3D mirror is activated in the light field display (e.g., when the user enters the 3D mirror zone). Figure 3 An example of presenting instructions to a user for activating 3D mirroring in a 3D mirroring zone according to some embodiments is shown.

[0100] In some embodiments, the 3D mirror area can be used to present a preview of the light field visualization of the mirrored 3D object. The initial size, orientation, and position (including depth) of the mirrored 3D object can be determined. For example, the orientation of the 3D content can be determined based on the orientation of the 3D object in the 2D display and based on the position of the viewer relative to the light field display. For example, for a 2D display displaying 3D content, the normal of the display can be determined in order to obtain the normal of the 3D content. The orientation of the mirrored 3D content can be calculated so that the normal of the 3D content is oriented toward the position of the viewer relative to the light field display. The relative position of the user can be detected, for example, using the captured RGB-D camera feed of the room. The determination of the initial size of the mirrored 3D object can be based on the size of the display and / or the viewing distance. In some embodiments, the size of the mirrored 3D content may depend on its relative size in the 2D display. For example, 3D content presented in full screen mode in a 2D display can be mirrored in a light field display and presented in full screen mode. In some embodiments, the initial size of the 3D object may depend on the detected viewing distance. For example, for longer viewing distances, the mirrored 3D object may be scaled larger to achieve better visibility. According to some embodiments, an exemplary scaling ratio calculation for the mirrored 3D object is Fig.16 Depicted in.

[0101] Figure 4 An example preview view of a mirrored 3D object 402 rendered in a light field display 404 is shown. The preview view may provide a low-quality light field visualization of the 3D object. The preview may provide guidance to the user, such as an indication 406 that the size of the 3D object can be adjusted and an indication 408 that the orientation of the 3D object can be adjusted. Where different mirroring zones are used, an indication 410 of which zone is being controlled may be displayed. In some embodiments, the user may perform motion and gesture-based adjustments to 3D mirroring parameters. For example, the user may increase the size of a 3D object by moving towards the center point of the light field display, or decrease the size of the object by moving away from the center point of the display.

[0102] After the 3D mirror is fully prepared, high-quality light field visualization of the mirrored 3D object can be provided. Figure 5 A 3D mirror zone is depicted according to some embodiments presenting a high quality light field visualization of a mirrored 3D object 502. As shown in the example, information about other 3D mirror zones (eg, zones presenting other viewing directions of a light field display) may be provided.

[0103] 1.2 Viewer Description

[0104] The viewer description describes a viewer detected in a room the light field display is located in. The viewer description may be used, for example, to control the orientation of content within the light field display.

[0105] The viewer description may include the following information: A viewer identifier field, which may include a unique identifier of the detected viewer. A viewer name field, which may include the name of the viewer detected in the room. A website address field for a 3D object viewing service, which may include a website address (e.g., a URL) of a 3D object viewing service for the viewer to view the 3D object. An identifier field for a 3D object, which may include an identifier of a 3D object to be mirrored to the light field display. The viewer description may also include information about the position of the viewer relative to the light field display and / or information about the position of the mobile device relative to a center point of the light field display.

[0106] 1.3 3D Mirror Area Description

[0107] The 3D mirror area description may include the following information: A Name field, which may include the name of the 3D mirror area (e.g., "3D mirror area A"). A Boundary of the 3D mirror area in FOV field, which may include the horizontal and vertical position and size of the 3D mirror area in the field of view of the light field display. A Maximum viewing distance field, which may include the maximum viewing distance of the named mirror area relative to the center point of the light field display.

[0108] 1.4 3D Display Description

[0109] The 3D display description may include the following information: A spatial resolution field for the display, which may provide the spatial resolution supported by the light field display. An angular resolution field for the display, which may provide the angular resolution supported by the light field display. For example, the angular resolution of a light field display may be 0.5 degrees or 2 views per degree. A horizontal FOV field for the display, which may describe the horizontal field of view (e.g., in degrees) at which images from the light field display may be displayed. A vertical FOV field for the display, which may describe the vertical field of view (e.g., in degrees) at which images from the light field display may be displayed. A best viewing direction field for the display, which may be provided. For example, the front side of the display may be defined as the best viewing direction for the light field display. The 3D display description may include one or more 3D mirror zone descriptions as previously described. For example, in the simplest case, one 3D mirror zone is provided for the light field display.

[0110] 1.5 Description of the designated mirror area

[0111] The named mirror region description may include the following information: A name field, which may include a descriptive name for the named mirror region for the 3D object (e.g., "front view" or "side view"). A preferred bounds field for the named mirror region in the FOV, which may include the preferred horizontal and vertical position and size of the named mirror region in the field of view. A maximum viewing distance field, which may provide the maximum viewing distance of the named mirror region relative to the center point of the light field display. A preferred size field for 3D objects, which may provide the preferred size of the 3D objects presented in the named mirror region in the light field display. A preferred orientation field for 3D objects, which may provide the preferred orientation of the 3D objects presented in the named mirror region in the light field display relative to the light field display. In some embodiments, the preferred orientation of the 3D object may be based on the relative position of the viewer or the mirror source. A preferred position field for 3D objects, which may provide the preferred position of the 3D objects presented in the named mirror region in the light field display.

[0112] 1.6 3D mirror model

[0113] The 3D mirror model may include the following information: an associated viewer field for a named mirror zone, which may identify the viewer for whom the 3D mirror model is provided. Since the orientation of a 3D object in a 3D mirror zone may depend on the relative position of the viewer mirroring the 3D content to the light field display, the associated viewer field for the named mirror zone may be used to establish a preferred orientation of the mirrored 3D object based on the relative position of the viewer. An identifier field for a 3D object, which may include an identifier of the 3D object to be mirrored to the light field display. A 3D mirror mode field, which may include a value for a 3D mirror mode, such as "limited 3D mirror mode" or "broadcast 3D mirror mode". In limited 3D mirror mode, the mirrored 3D content is configured to be visible only to viewers of the corresponding zone. In broadcast 3D mirror mode, the 3D content is mirrored to all 3D mirror zones of the light field display. A size field for a 3D object in a 2D display, which may describe the size of a 3D object in a 2D display relative to the full 2D display size. This information may be used to display the mirrored 3D object on the light field display at a specific initial size. For example, a 3D object may be displayed so that it appears to a viewer to maintain the same size when mirrored. An orientation field for 3D objects in a 2D display, which may include the orientation of the 3D object projected onto the 2D display. This information may be used to enable a mirrored 3D object to be rotated to a specific orientation. For example, a mirrored 3D object may be displayed so that it appears to a viewer to maintain the same orientation when mirrored. A low-quality 3D model field for an object, which may describe a low-quality 3D model for a 3D object to be mirrored onto a light field display. For example, the 3D model may be used to provide a preview for the 3D object in a light field display. A high-quality 3D model field for an object, which may describe a high-quality 3D model for a 3D object to be mirrored onto a light field display. For example, the 3D model may be used to provide a high-quality light field visualization of the 3D object. A named mirror region description field, which may contain one or more named mirror region descriptions for a 3D object to be mirrored onto a light field display as previously described.

[0114] 2. Example Architectures of Some Implementation Scenarios

[0115] Figure 6 is a schematic diagram of an exemplary environment for mirroring 3D objects from a mobile device to a light field display according to some embodiments. Figure 6As shown in the example of , environment 600 includes a mobile device 602 with an Internet connection. Mobile device 602 includes a browser 604 and a 2D display 606. Browser 604 can be a software module capable of obtaining 3D content from a server side (e.g., a content server 608 with a 3D visualization service 610) and representing the 3D content in a 2D display of the mobile device. 2D display 606 can be used to output visual content from the mobile device. Environment 600 also includes a light field display 612 with an Internet connection. Light field display 612 includes a 3D mirroring service module 614, a light field visualization module 616, and a light field display module 618. 3D mirroring service 614 can be a software module that coordinates the mirroring of a 3D object to a light field display. Light field visualization 616 can be a software module that operates to generate light field visualization, for example, for a 3D model of a mirrored 3D object. Light field display 618 is used to output light field visualization.

[0116] In some embodiments, the relative orientation of the viewer may be used to control the orientation of the 3D content mirrored to the light field display. Utilizing the relative orientation between the viewer and the corresponding display may require the ability to obtain this data. In some embodiments, the light field display operates to obtain the relative orientation of the viewer. In some embodiments, the smart space is used to obtain information about the relative orientation of the viewer. An internet connection may be used to enable a module of the light field display to obtain this data from the smart space for 3D mirroring. Figure 6 The depicted environment 600 includes a smart space 620 having a viewer tracking module 622, a room analysis module 624, an indoor scene analysis module 626, and an RGB-D camera 628. The RGB-D camera 628 may include a sensor that operates to provide an RGB-D (color video and depth) video feed of viewers in the room. The viewer tracking module 622 is a service that operates to track the positions of different viewers in the room, for example, by using the video feed of the RGB-D camera 628. In some embodiments, the viewer tracking module 622 can track the position of the mobile device of the tracked viewer in the room. The room analysis module 624 can be operated to prepare a room model for viewing the environment. The indoor scene analysis module 626 can be operated to interpret the main surfaces, objects, and supporting relationships of the indoor scene from the RGB-D image. The indoor scene analysis module 626 can operate using existing indoor scene analysis methods.

[0117] Figure 6An exemplary information flow between components in environment 600 is also depicted. For example, at 630, mobile device 602 sends a 3D visualization request to content server 608. At 632, the content server provides a 3D visualization response to the mobile device. At 634, a 3D mirror mode is provided from the mobile device to the content server. At 636, light field display 612 sends a 3D mirror analysis request to smart space 620. In some embodiments, the 3D mirror analysis request includes a 3D display description. At 638, the smart space provides a 3D mirror analysis response to light field display 612. In some embodiments, the 3D mirror analysis response includes a 3D mirror zone description. At 640, the smart space provides user notifications (e.g., including a viewer description) to light field display 612. At 642, the smart space may provide motion and gesture detection information to the light field display. The motion and gesture detection information may include, for example, a user ID, a detected motion, and / or a detected gesture. At 644, the light field display 612 sends a 3D mirror request to server 608. In some embodiments, the 3D mirroring request includes a viewer description and a 3D display description. At 646, the content server 608 provides a 3D mirroring response to the light field display. In some embodiments, the 3D mirroring response includes a 3D mirroring model. At 648, the light field display sends a complete 3D model request to the server 608, which may include, for example, a 3D object identifier. At 650, the server provides a complete 3D model response to the light field display 618.

[0118] 3. Exemplary Process

[0119] Fig. 7A An overview of a process for mirroring a 3D object into a field of view of a light field display according to some embodiments is depicted. In some embodiments, the process includes providing a quick preview of the mirrored object in the light field display. In some embodiments, the process includes motion and gesture based adjustment of 3D mirroring parameters.

[0120] See also Fig. 7A , the exemplary process 700 includes a set of sub-processes: at 710, activating the mirroring of the 3D object; at 720, preparing the mirroring of the 3D object; and at 730, outputting the mirrored 3D object. These sub-processes are described in more detail in the following sections.

[0121] 3.1 Activate mirroring of 3D objects

[0122] Figure 7BAn exemplary subprocess for activating mirroring of a 3D object according to some embodiments is depicted. In this example, subprocess 710 includes: at 712, activating 3D mirror mode; at 714, deploying a 3D mirroring zone; at 716, detecting a user and a position of the user relative to a light field display; and at 718, triggering 3D mirroring.

[0123] At 712, subprocess 710 includes activating a 3D mirror mode. In some embodiments, the 3D mirror mode may be set to, for example, the "limited 3D mirror mode" or the "broadcast 3D mirror mode" previously described. For example, a user with a mobile device / goggles may select a 3D object of interest in the 3D visualization service. The user may activate 3D mirroring by setting the 3D visualization service mode.

[0124] At 714, subprocess 710 includes deploying a 3D mirror area, which may include delivering a 3D mirror analysis request to a room analysis service through a 3D mirror service. The room analysis service may use an RGB-D video feed to detect room geometry and visual obstacles that affect viewing of a light field display in a room. In some cases, the room geometry may impose restrictions on possible viewing distances and viewing directions. The room analysis service may generate a room model to define possible viewing directions and viewing distances of a light field display in a room. The room analysis service may use the room model and the 3D display description to determine the configuration of the 3D mirror area, such as determining the optimal orientation to maximize visibility of the 3D mirror area in the room. The room analysis service may return the configured 3D mirror area in a 3D mirror analysis response. Preparation of the room model may be based on existing indoor scene analysis solutions, 3D object detection methods, and / or RGB-D camera feeds. Scene analysis solutions that may be employed in some embodiments are described in N. Silberman et al., “Indoor segmentation and support inference from RGBD images”, European Conference on Computer Vision, Springer, Berlin, Heidelberg, 2012, pp. 746-760; and S. Song et al., “Deep sliding shapes for amodal 3D object detection in RGB-D images”, Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, 2016, pp. 808-816.

[0125] At 716, subprocess 710 includes detecting a user and a position of the user relative to the light field display, which may include the use of a user tracking module. The user tracking module may use an RGB-D camera feed to identify the user, for example, when the user walks into a 3D mirror area of ​​the light field display. The user tracking module may detect the position of the user relative to the light field display. The user tracking service may also detect the position of the mobile device relative to the center point of the light field display. The user tracking service may generate a viewer description for the identified user and deliver a user notification message containing the viewer description of the 3D mirror service. Detecting the user and the user's position may be based on an RGB-D camera feed and / or an existing skeleton tracking solution such as that described in the following document: A. Kar et al., "Skeletal tracking using Microsoft Kinect", Methodology, 2010, 1.1:11, or a head tracking solution such as that described in the following document: Czupryński and Strupczewski, "High accuracy head pose tracking survey", International Conference on Active Media Technology, Springer, Cham, 2014, pp. 407-420. For example, a solution that supports skeleton tracking in a room such as a Kinect sensor can be used, which combines several advanced sensing hardware including a depth sensor, a color camera, and a four-microphone array that provides full-body 3D motion capture, facial recognition, and voice recognition capabilities, such as the solution described in the following document: Z. Zhang, "Microsoft Kinect sensor and its effect", IEEE Multimedia, 2012, 19.2:4-10.

[0126] At 718, subprocess 710 includes triggering 3D mirroring. In some embodiments, the 3D mirroring service may use the user identity defined in the viewer description to determine a website address based on the user identity. The website address may provide access to a 3D visualization service for viewing 3D objects on a mobile device. In addition, a link to the 3D content may be shared for the 3D mirroring service. The 3D mirroring service may use the 3D mirroring zone description of the 3D display description and the position of the viewer relative to the light field display defined in the viewer description to trigger 3D mirroring to the light field display.

[0127] In some embodiments, triggering 3D mirroring 718 employs proximity-based triggering. In the case of proximity-based triggering of 3D mirroring, a 3D object is automatically mirrored to a designated 3D mirroring area of ​​a light field display when a user enters the light field display. For example, 3D mirroring may be triggered when the distance between the user and the display is less than a defined threshold.

[0128] In some embodiments, triggering 3D mirroring 718 employs a tag-based trigger. In the case of a tag-based trigger of 3D mirroring, the 3D mirroring service may detect the tag using a captured camera feed of the environment and begin 3D mirroring the 3D object defined in the tag. For example, the 3D mirroring service may detect a QR code from a mobile display device of a user approaching a light field display and, in response, begin 3D mirroring the object defined in the QR code.

[0129] In some embodiments, triggering 3D mirroring 718 employs a proximity and device location based trigger. In the case of proximity and device location based triggers, the 3D mirroring service may activate a "drop object" mode when the user has entered a designated 3D mirroring area of ​​the light field display (e.g., when the distance between the user and the display is less than a defined threshold). When the "drop object" mode is activated, instructions for transferring or "dropping" a 3D object from the mobile device to the light field display may be displayed.

[0130] Fig. 9 An exemplary scenario is shown in which a user according to some embodiments utilizes proximity-based and device-position-based triggering of 3D mirroring to transfer a 3D object from a mobile device to a light field display. In this example, the user places his / her mobile device above the light field display and waits for three seconds ("casting action A"). Therefore, the object shown in the display of the mobile device is "cast" to the light field display and is shown as a 3D object in the correct orientation, size, and position in the designated 3D mirror area. Then, the user moves the mobile device to the right, turns the mobile device upside down, and waits for three seconds ("casting action B"). The viewer tracking module detects the device position and triggers 3D mirroring based on the new position. Therefore, the object shown in the display of the mobile device is "cast" to the light field display and is shown as a 3D object in the correct orientation (inverted position), size, and position in the designated 3D mirror area. In some embodiments, an animation effect is provided in response to the casting action. For example, the "casting of 3D objects" is visualized for the user on a 2D display and a light field display.

[0131] 3.2 Preparing the mirror image of the 3D object

[0132] Figure 7CAn exemplary subprocess for preparing a mirror image of a 3D object according to some embodiments is depicted. In this example, subprocess 720 includes: determining an initial size, orientation, position, and 3D mirror image model of the 3D object at 722; preparing a preview of the mirrored 3D object at 724; displaying instructions for adjusting 3D mirror image parameters at 726; and performing gesture and motion based adjustments to the designated mirror image area at 728.

[0133] At 722, subprocess 720 includes determining an initial size, orientation, position, and 3D mirror model of the 3D object, which may employ a 3D mirror service. In some embodiments, the 3D mirror service sends a 3D mirror request to the 3D visualization service. The 3D visualization service may determine the 3D mirror model and deliver the 3D mirror model to the 3D mirror service in a 3D mirror response.

[0134] Determining a 3D mirror model may include the following steps. The 3D visualization service may determine an identifier of an associated viewer (e.g., a user who mirrors 3D content to a light field display) who has specified a mirror area. The 3D visualization service may determine an identifier of a 3D object, a low-quality 3D model of the 3D object, and optionally a high-quality 3D model of the 3D object. The 3D visualization service may determine information about the size and orientation of the 3D object in the 2D display. The 3D visualization service may determine a 3D mirror mode selected by the user. The value of the 3D mirror mode may be, for example, "limited 3D mirror mode" or "broadcast 3D mirror mode". The 3D visualization service may use the 3D display description and select a 3D mirror area that has not been assigned to other users. By using the viewer's position and the 3D mirror area description, the 3D visualization service may, for example, select a 3D mirror area oriented toward the user's current position. In "limited 3D mirror mode", content may be displayed only to associated viewers. In "broadcast 3D mirror mode", 3D content may be limited to mirroring to all 3D mirror areas that have not been assigned to other users.

[0135] One or more named mirror areas may be selected for 3D mirroring of a 3D object. The named mirror areas may be selected, for example, based on the size of the 3D mirror area. It may be useful to consider the type and shape of the 3D object and the amount of available space for presenting the named mirror areas in the 3D mirror area. For example, a named mirror area may be provided for each of the front view and the side view of a car. For a flat shaped object such as a 3D map, a named mirror area may be provided only for the front view.

[0136] In some embodiments, the initial size of the mirrored 3D content is determined based on its relative size in the 2D display. For example, 3D content presented in full screen mode on a 2D display can be mirrored in a light field display and presented in full screen mode. In some embodiments, the initial size of the 3D object is determined based on the distance between the viewer and the center point of the light field display. For example, for longer viewing distances, the size of the 3D object can be set larger to achieve better visibility.

[0137] The orientation of the 3D object in the designated mirrored area may depend on the relative positions of the 3D model, the mirrored area, and the user. In some embodiments, the orientation of the 3D object in the 2D display is used to determine the initial orientation of the 3D object, for example by setting the mirrored 3D object to the same orientation as it is presented in the 2D display. For example, for a 2D display showing 3D content, the normal of the display may be determined in order to obtain the normal of the 3D content. The orientation of the mirrored 3D content may be calculated so that the normal of the 3D content is oriented toward the position of the viewer relative to the light field display. In some embodiments, the user may change the orientation of the 2D display in order to project the 3D object to the light field display in a specific desired orientation. Therefore, the orientation of the 2D display may be used to determine the orientation of the mirrored 3D object in the light field display.

[0138] In some embodiments, the orientation of the 3D object can be determined based on the named mirror area. The 3D model can, for example, define the front view, side view, top view, and back view directions of the object. These named mirror areas corresponding to the viewing direction of the 3D object can be used to provide an orientation specified for the 3D object for the relative position of the user.

[0139] Determination of an initial position of the 3D object may be based on detection of clipping of the mirrored 3D object in an angled view of the 3D mirror zone. The position (eg, depth of the 3D object in a light field visualization) may be initially adjusted to minimize clipping of the 3D object.

[0140] At 724, subprocess 720 includes preparing a preview of the mirrored 3D object. In some embodiments, the 3D mirror service uses the 3D mirror model and prepares a low-quality source view for light field visualization of the named mirror area defined in the 3D mirror model. It may be preferred to enable the preparation of a preview light field visualization of the mirrored 3D object in a very short time. To achieve this, the 3D mirror service may be operable to accelerate 3D rendering to prepare the source view for light field visualization more quickly. For example, the 3D mirror service may first use a simplified 3D model of the 3D object to minimize the rendering time of the source view for light field visualization. The 3D mirror service may later download the complete 3D model of the 3D object. In addition, the 3D mirror service may use a reduced number of polygons when rendering. In some embodiments, textures may be ignored when preparing a preview of the mirrored 3D object. Textures may be downloaded later and used for rendering to provide high-quality visualization of the mirrored 3D object.

[0141] At 726, subprocess 720 includes displaying instructions for adjusting 3D mirroring parameters. In some embodiments, the 3D mirroring service provides instructions to the user for mirroring 3D objects to the display. For example, the provided instructions may instruct the user how to adjust the 3D objects mirrored in the light field display. The viewer tracking service may deliver information about the detected movements and gestures of the user to the 3D mirroring service.

[0142] At 728, subprocess 720 includes performing gesture and motion based adjustments to the named mirror region. The 3D mirror service may use information about the user's gestures and motions to adjust 3D mirror parameters associated with the current named mirror region. The user may use motion in the named mirror region to adjust the size of the 3D object shown in the light field visualization preview, for example, by moving toward or away from the display. The user may use lateral motion in the named mirror region, for example, to adjust the orientation of the mirrored 3D object in the light field display. The user may perform a gesture (e.g., a "thumbs up" gesture), for example, to accept the defined 3D object mirror parameters. Techniques for detecting gestures that may be implemented in some embodiments include those described in Gupta and Suwei, “Gesture-based interaction and communication: automated classification of hand gesture contours,” IEEE Transactions on Systems, Man, and Cybernetics, Part C (Applications and Reviews), 2001, 31.1: 114-120; and Hürst and Van Wezel, “Gesture-based interaction via finger tracking for mobile augmented reality,” Multimedia Tools and Applications, 2013, 62.1: 233-258.

[0143] 3.3 Output mirrored 3D objects

[0144] Fig.7D Depicted is an exemplary subprocess for outputting a mirrored 3D object according to some embodiments. In this example, subprocess 730 includes: at 732, preparing a full light field visualization of the mirrored 3D object; and at 734, outputting a light field visualization of the mirrored 3D object.

[0145] At 732, subprocess 730 includes preparing a full light field visualization of the mirrored 3D object. In some embodiments, the 3D mirroring service obtains a high-quality 3D model of the mirrored 3D object. The 3D mirroring service can use the 3D model and render a high-quality source view for light field visualization to be presented in a 3D mirror area of ​​the light field display at a selected orientation, position, and scale.

[0146] At 734, subprocess 730 includes outputting a light field visualization of the mirrored 3D object. In some embodiments, the 3D mirror service displays the light field visualization of the mirrored 3D object and the 3D mirror area in the light field display. The 3D mirror service may also provide information about other 3D mirror areas available in other viewing directions in the light field display.

[0147] 3.4 Additional Exemplary Methods

[0148] In some embodiments, the process includes performing a fast preview of the mirrored object in the light field display. In some embodiments, the process includes motion and gesture based adjustment of 3D mirroring parameters.

[0149] FIG. 8A to FIG. 8BA sequence diagram of a process for mirroring a 3D object to a field of view of a light field display according to some embodiments is formed. The figure depicts a user 802, a mobile device 804 with a browser 806, a server 808 with a 3D visualization service 810, a light field display 812 with a 3D mirroring service 814, and a smart space 816 with a room analysis service 818 and a viewer tracking service 820. At 822, the user interacts with a user interface (UI) of the mobile device. For example, the user can select a 3D mode and / or a 3D object of interest in the 3D visualization service, for example, using the mobile device 804. At 824, the mobile device 804 sends a request for 3D visualization to the server-side 3D visualization service 810. At 826, the mobile device receives a response to the 3D visualization from the 3D visualization service 810. At 828, the 3D visualization service receives a 3D mirroring mode, such as a 3D mode determined or selected at the mobile device. At 830, the 3D mirroring service 814 sends a 3D mirroring analysis request to the room analysis service 818. The 3D mirroring analysis request may, for example, include a 3D display description. At 832, the room analysis service 818 performs room analysis. At 834, the room analysis service 818 determines the configuration of the 3D mirroring area, for example, to determine the best orientation to maximize the visibility of the 3D mirroring area in the room. The configuration of the 3D mirroring area may determine, for example, a room model and / or a display description. At 836, the room analysis service returns the configured 3D mirroring area in a 3D mirroring analysis response, which may include a 3D mirroring area description. At 838, the viewer tracking service 820 delivers a user notification message to the 3D mirroring service, which may include, for example, a viewer description of the identified tracking user. At 840, the 3D mirroring service triggers 3D mirroring. Triggering 3D mirroring may be based on information included in the viewer description, 3D display description, and / or 3D mirroring area description as previously described. At 842, the 3D mirroring service 814 sends a 3D mirroring request to the 3D visualization service. The 3D mirroring request may include viewer description and / or 3D display description information. At 844, the 3D visualization service determines a 3D mirror model, and at 846 delivers a 3D mirror response, which includes, for example, the 3D mirror model. The 3D model may be determined, for example, based on a 3D mirror display description and a 3D model of an object to be mirrored. At 848, the 3D mirror service 814 prepares a preview of the mirrored 3D object, for example, based on the 3D mirror model. At 850, the 3D mirror service 814 displays instructions to the user, for example, to guide the user on how to adjust the 3D object mirrored in the light field display by adjusting 3D mirror parameters. At 852, the viewer tracking service 820 provides motion and gesture detection information to the 3D mirror service 814. At 854, the 3D mirror service 814 performs adjustments to the designated mirror area (e.g., to adjust the preferred size, orientation, position, etc. of the 3D object).At 856, the 3D mirroring service 814 sends a complete 3D model request to the 3D visualization service 810, which may include a 3D object identifier. At 858, the 3D visualization service 810 provides a complete 3D model response to the 3D mirroring service, which includes, for example, a complete (e.g., high-quality) 3D model. At 860, the 3D mirroring service prepares a full light field visualization of the mirrored 3D object. At 862, the 3D mirroring service outputs the mirrored 3D object, for example, for visual display.

[0150] According to some embodiments, Figure 6 and FIG. 8A to FIG. 8B The locations of the modules depicted are provided as examples only. The locations of the modules may vary; for example, modules for viewer and gesture tracking may be deployed to a light field display or a smart space where a light field display is located.

[0151] Fig.10 An example of mirroring a 3D object from a personal display to a group display while maintaining the orientation and scaling of the 3D object from the viewer's perspective is depicted according to some embodiments. A virtual object 1002 is displayed on a 2D display device 1004, which may be a personal display device, such as a phone or tablet. The tablet provides parameters 1006 for displaying another version 1008 of the virtual object on a 3D display, which may be a light field display or other autostereoscopic display. The orientation of the virtual object 1008 on the 3D display may be selected based at least in part on the orientation of the virtual object 1002 on the 2D display and the position of the user 1012. For example, the orientation of the virtual object 1008 on the 3D display may be selected so that the virtual objects 1002 and 1008 have substantially the same orientation relative to the user.

[0152] Fig.11 is a sequence diagram of an exemplary process for mirroring 3D content from an originating device to a light field display or other autostereoscopic display according to some embodiments. Fig.11In the method, an initial device 1102 (e.g., a mobile phone) having an initial display tracks the relative size and position of the viewer position and the second display of the second device (e.g., a light field display 1104). At 1106, a SLAM model is constructed. At 1108, a 3D mirror mode is selected. For example, a user can initiate mirroring or transmission of content from the initial device to the second device by selecting the mirror mode. Based on the selected mode, parameters describing the scaling and orientation to be used to present the content on the second display are calculated. The calculated scaling factor (relative to the size of the second display) can be selected to present the viewer with content of the same apparent size as that seen on the smaller but closer initial display. The orientation to be used on the second display can be selected so that the orientation of the object seen by the viewer on the second display is the same as the orientation of the object seen when viewed on the initial display. At 1112, a content descriptor (e.g., a URL for obtaining content) is shared with the second device. At 1114, viewing parameters (e.g., scaling parameters and viewing orientation parameters) are shared with the second display. The content descriptor and viewing parameters may be sent together (e.g., as a single content description) or separately. At 1116, the second device retrieves the content (e.g., via the Internet). At 1118, the second device appropriately sets the scaling and viewing orientation before rendering to the second display. At 1120, the light field image is rendered and displayed.

[0153] Fig.12 1 is a sequence diagram of a second exemplary process for mirroring 3D content from an initial device to a light field display according to some embodiments. At 1206, 3D content is selected at the initial device. At 1208, the light field display 1204 tracks the position of the viewer and the initial device. At 1210, the light field display calculates a relative orientation and a scaling factor. At 1212, a content descriptor (e.g., a URL for obtaining content) is shared with a second device. At 1214, viewing parameters may be shared with a second light field display. At 1216, the light field display 1204 obtains the content (e.g., via the Internet). At 1218, the light field display appropriately sets the scaling factor and viewing orientation before rendering to the second display. At 1220, a light field image is rendered and displayed.

[0154] 4. Example use cases

[0155] exist FIG. 13A to FIG. 13D In the example shown, different 2D representations 1304a-1304d of a virtual 3D object 1306 (in this example, a virtual model of a car) are displayed on a 2D display device 1302. The object orientation of the virtual 3D object 1306 is different from the 2D representation 1304a-1304d of the virtual 3D object 1306. FIG. 13A to FIG. 13DIn this example, the object orientation of the virtual 3D object relative to the 2D display device can be expressed relative to a coordinate system, where x 2D The y axis extends to the right side of the 2D display device. 2D The z axis extends toward the top of the 2D display device, and the 2D The axis extends perpendicular to the surface of the 2D display device in a direction toward the user of the display device. The object orientation of the virtual 3D object 1306 may be characterized using other angles (such as Euler angles or altitude and azimuth) or by other techniques (such as a vector or point on a unit sphere). In this example, the object orientation of the virtual 3D object is represented by a rotation around the y axis. 2D The rotation angle β about the axis is characterized. 2D The principles explained for rotations about an axis can be applied similarly to rotations about other axes or in other directions.

[0156] exist Fig.13A In the example of , the virtual 3D object 1306 is displayed as a 2D representation 1304a having an object orientation characterized by an angle β=0°. Fig. 13B In the example of , the virtual 3D object 1306 is displayed as a 2D representation 1304b having an object orientation characterized by an angle β=90°. Fig. 13C In the example of , the virtual 3D object 1306 is displayed as a 2D representation 1304c having an object orientation characterized by an angle β=180°. Fig.13D In the example of FIG. 1 , the virtual 3D object 1306 is displayed as a 2D representation 1304 d having an object orientation characterized by an angle β=270°.

[0157] A user of a 2D display device such as device 1302 may provide instructions (eg, by selecting an icon) to mirror a virtual 3D object to a 3D display system, such as a light field display or other autostereoscopic display.

[0158] Fig.14A One such example of a mirroring process is shown in Fig.14A In the example of FIG. 1 , a user 1400 of a 2D display device 1302 is viewing a representation 1304a of a virtual 3D object 1306 having an object orientation characterized by an angle β=0°, which gives the user a perspective view of the right side of a car represented by the virtual 3D object 1306. The user 1400 enters an instruction to mirror the virtual 3D object 1306 to the 3D display system 1402 (in Fig.14A In response to the user instruction, the 2D display device 1302 sends information identifying the virtual 3D object and information identifying the orientation of the object at the 2D display device to the 3D display system 1402.

[0159] The information identifying the virtual 3D object may include 3D model information, such as one or more polygon meshes and texture information, or the information identifying the virtual 3D object may include a link or other address that a 3D display system may use to obtain the 3D model information.

[0160] exist Fig.14A In an example of 3D display system 1402, 3D display system 1402 is integrated into a single display device. In some embodiments, the display system may include separate components, such as a separate display device and controller. In such embodiments, instructions for mirroring virtual 3D objects may be received by the controller (or other components) of the 3D display system, rather than by the display device itself.

[0161] exist Fig.14A In the example of , the 3D display system 1402 receives information identifying the virtual 3D object 1306 and information identifying the object orientation used by the 2D display device, which in this example indicates β=0°.

[0162] The 3D display system determines the viewing direction of the user 1400 from the 3D display device of the 3D display system 1402 to the 2D display device. In some embodiments, the 3D display system determines the viewing direction by receiving a direction from an external entity (such as, from a SLAM system or from a 2D display device 1302). In other embodiments, the 3D display system operates to determine the viewing direction of the user, for example, using a user of a camera. In some embodiments, the position of the user 1400 is inferred based on the position of the 2D display device. For example, the 2D display device may transmit an RF signal that allows the device to be positioned. In some embodiments, the position of the 2D display device may be used as the position of the user. In other embodiments, the position of the user may be inferred from the position of the 2D display device, for example, by assuming that the user's eyes are 50 cm (or some other distance) above or in front of the screen of the 2D display device.

[0163] In some embodiments, the user's location may be determined based on input from the user. For example, the user may provide input (either directly to the 3D display system or through a 2D display device) of the mirror zone the user is in or the zone the user wishes to use. The 3D display device may inform the user which zone the user is occupying by displaying different zone identifiers in different mirror zones, or may use other information to allow the user to select a zone.

[0164] The 3D display system 1402 determines an object orientation for displaying the virtual 3D object 1306. The orientation may be determined based on the object orientation transmitted by the 2D display device and the viewing direction of the user. Additional factors may be considered in some embodiments. Fig.14AIn the example of , the object orientation is selected so that the apparent orientation of the 3D representation viewed from the position of the user 1400 corresponds to the first object orientation as displayed on the 2D display device. For example, because the 2D display device is showing the right side of the virtual car, the 3D display device will display the virtual car with its right side facing the user 1400. The 3D display system then displays the 3D representation of the virtual 3D object 1306 on the 3D display device using the determined object orientation.

[0165] exist Fig.14A In the example of , the viewing direction from the 3D display device to the user 1400 is characterized by an angle α. In order to provide an object orientation such that the apparent orientation of the 3D representation (as seen by the user 1400) corresponds to the object orientation displayed on the 2D display device, the 3D display system combines the orientation angle β received from the 2D display device and the angle α representing the viewing direction angle to give a display angle θ for displaying the virtual object 1306 on the 3D display device 1402. Fig.14A In the example of , angles α and β are combined by addition so that θ=α+β. In this example, θ, α, and β all represent counterclockwise rotation, as seen from above. (The viewer 1400 is displaced from the normal of the 3D display in a clockwise manner, giving a negative viewing direction angle.) However, other notations or coordinate systems may be used in other embodiments. The resulting orientation of the virtual object 1306 is Fig.14A is schematically shown as a plan view and in Fig. 14B Schematically shown as a perspective view.

[0166] Fig. 14B is a perspective view of display device 1402 from the perspective of user 1400. Fig. 14B As shown, from the user's perspective, the apparent orientation of 3D representation 1404 corresponds to the object orientation displayed as 2D representation 1304a on the 2D display. If the object orientation has not been rotated by an amount corresponding to viewing direction angle α, then from the user's perspective, the user's view of 3D representation 1404 would be a view from the right rear of the virtual car, and therefore would not correctly mirror the view displayed on the 2D display.

[0167] Fig.15A Another example of the mirroring process is shown in Fig.15A In the example of FIG. 1 , a user 1400 of a 2D display device 1302 is viewing a representation 1304b of a virtual 3D object 1306 having an object orientation characterized by an angle β=90°, which gives the user a perspective view of the rear of a car represented by the virtual 3D object 1306. The user 1400 enters an instruction to mirror the virtual 3D object 1306 to the 3D display system 1402 (in Fig.15AIn response to the user instruction, the 2D display device 1302 sends information identifying the virtual 3D object and information identifying the orientation of the object at the 2D display device to the 3D display system 1402.

[0168] exist Fig.15A In the example of , the 3D display system 1402 receives information identifying the virtual 3D object 1306 and information identifying the object orientation used by the 2D display device, which in this example indicates β=90°.

[0169] The 3D display system determines the viewing direction of the user 1400 from the 3D display device of the 3D display system 1402 to the 2D display device, and determines the object orientation for displaying the virtual 3D object 1306. The orientation may be determined based on the object orientation transmitted by the 2D display device and the viewing direction of the user. Fig.15A In the example of , the object orientation is selected so that the apparent orientation of the 3D representation viewed from the position of the user 1400 corresponds to the object orientation as displayed on the 2D display device. For example, because the 2D display device is showing the rear of the virtual car, the 3D display device will display the virtual car with its rear facing the user 1400. The 3D display system then displays the 3D representation of the virtual 3D object 1306 on the 3D display device using the determined object orientation.

[0170] exist Fig.15A In the example of , the viewing direction from the 3D display device to the user 1400 is characterized by an angle α. In order to provide an object orientation such that the apparent orientation of the 3D representation (as seen by the user 1400) corresponds to the object orientation displayed on the 2D display device, the 3D display system combines the orientation angle β received from the 2D display device and the angle α representing the viewing direction angle to give a display angle θ for displaying the virtual object 1306 on the 3D display device 1402. The resulting orientation of the virtual object 1306 is Fig.15A is schematically shown as a plan view and in Fig. 15B Schematically shown as a perspective view.

[0171] Fig. 15B is a perspective view of display device 1402 from the perspective of user 1400. Fig. 15B As shown, from the user's perspective, the apparent orientation of the 3D representation 1504 corresponds to the object orientation displayed as 2D representation 1304b on the 2D display. If the object orientation is only rotated by angle β=90°, but not by an amount corresponding to the viewing direction angle α, then from the user's perspective, the user's view of the 3D representation 1504 will be a view from the left rear of the virtual car, and therefore will not correctly mirror the view displayed on the 2D display.

[0172] In some embodiments, once mirroring begins, the 2D display device may receive user input that changes the orientation of the object displayed at the 2D device. In response to the user input, the 2D display device may send information identifying the change in the orientation of the first object to the 3D display system. The 3D display system may receive information identifying the change in the orientation of the first object, modify the 3D object orientation according to the change in the orientation of the first object, and display the 3D representation of the virtual 3D object on the 3D display device using the modified second object orientation. For example, if the object is initially displayed by the 3D display at a display angle θ=α+β, and the user of the 2D device locally changes the object orientation from β to β+Δβ, the display angle on the 3D display device may be changed to θ=α+β+Δβ. Similarly, if the user of the 2D device changes the position relative to the 3D display device, in some embodiments, the display angle on the 3D display device may be changed to θ=α+β+Δα. However, in other embodiments, the display angle on the 3D device may remain unchanged when the user changes position, thereby allowing the user to "walk around" around the representation once the original orientation of the 3D representation has been determined.

[0173] In some embodiments, the mirroring operation includes controlling the size of the 3D representation of the virtual object. Examples of such embodiments are described in Fig.16 In some embodiments, in response to the user mirroring instruction, the 2D display device sends scaling information to the 3D display device. The scaling information may include information indicating the display size of the 2D representation on the 2D display device. In such embodiments, the 3D display system may determine the display size of the virtual 3D object based at least in part on the received display size. The 3D representation of the virtual 3D object may be displayed on the 3D display device using the determined display size.

[0174] In some embodiments, the 2D display system may determine a first viewing distance from the 2D display device to a user of the 2D display device. In such embodiments, the display size on the 3D display device may be inversely proportional to the first viewing distance. In some embodiments, the 3D display system may determine a second viewing distance from the 3D display device to a user of the 2D display device. In such embodiments, the display size on the 3D display device may be proportional to the second viewing distance.

[0175] 5. Other implementation plans

[0176] An exemplary method for mirroring a 3D object from a first display to a second display according to some embodiments may include: identifying a relative orientation and a relative size of the first display and the second display; selecting a 3D mirror mode; calculating viewing parameters based on the relative orientation, the relative size and the 3D mirror mode for rendering on the second display; and transmitting a description of the 3D content and the viewing parameters to the second display.

[0177] For some embodiments of the example methods, the second display may be a light field display.

[0178] For some embodiments of the exemplary method, the second display may be an autostereoscopic display.

[0179] For some embodiments of the exemplary method, the first display may be a 2D display.

[0180] For some embodiments of the exemplary method, the first display may be a handheld display.

[0181] For some embodiments of the exemplary methods, the object parameters may include orientation parameters.

[0182] For some embodiments of the exemplary method, the object parameters include scaling parameters.

[0183] For some embodiments of the example method, the determination of the relative position may be accomplished based on a SLAM model generated at the first display, and the position of the second display is identified within the SLAM model.

[0184] For some embodiments of the exemplary method, the determination of relative position may be accomplished by a smart space.

[0185] For some implementations of the exemplary method, a position of a viewer relative to the first display and the second display may be detected and used to calculate the orientation parameter and the scaling parameter.

[0186] For some embodiments of the example method, the mirror mode may indicate that the apparent size of the object perceived by the viewer on the first display and the second display is equal, so that the object appears to be the same size on the first display and the second display.

[0187] For some embodiments of the exemplary method, the mirror mode may indicate that the orientation of an object perceived by the viewer on the first display and the second display is equal, so that the viewer sees the same side of the object when the object is on the first display and the second display.

[0188] Another exemplary method for mirroring a 3D object from a first display to a second display according to some embodiments may include: receiving a description of 3D content to be displayed from the first display; receiving viewing parameters from the first display to describe the display of the 3D content; receiving 3D content defined by the description of the 3D content; changing a representation of the 3D object based on the viewing parameters; and rendering a light field visualization for the changed representation of the 3D content.

[0189] For some embodiments of another exemplary method, the second display may be a light field display.

[0190] For some embodiments of another exemplary method, the second display may be an autostereoscopic display.

[0191] For some embodiments of another exemplary method, the first display may be a 2D display.

[0192] For some embodiments of another exemplary method, the first display may be a handheld display.

[0193] For some implementations of another exemplary method, the description of the 3D content may include a URL.

[0194] For some embodiments of another exemplary method, the viewing parameter may include an orientation parameter.

[0195] For some embodiments of another exemplary method, the viewing parameter may include a zoom parameter.

[0196] For some embodiments of another exemplary method, the determination of the relative position may be accomplished via the second display.

[0197] For some embodiments of another exemplary method, a second display may retrieve content related to the specified orientation.

[0198] An exemplary method for mirroring a 3D object from a first display to a second display may include: receiving a description of 3D content to be displayed from the first display; receiving parameters defining a view of the content on the first display from the first display; tracking a relative position of the first display and the second display; receiving 3D content defined by the description of the 3D content; changing a representation of the 3D object based on the viewing parameters; and rendering a light field visualization for the changed representation of the 3D content.

[0199] For some embodiments of the example methods, the second display may be a light field display.

[0200] For some embodiments of the exemplary method, the second display may be an autostereoscopic display.

[0201] For some embodiments of the exemplary method, the first display may be a 2D display.

[0202] For some embodiments of the exemplary method, the first display may be a handheld display.

[0203] For some implementations of the example method, the description of the 3D content may include a URL.

[0204] For some embodiments of the exemplary methods, the viewing parameter may include an orientation parameter.

[0205] For some embodiments of the exemplary method, the viewing parameter may include a zoom parameter.

[0206] For some embodiments of the exemplary method, the determination of the relative position may be accomplished via the second display.

[0207] For some implementations of the example method, the second display may retrieve content related to the specified orientation.

[0208] In some embodiments, a method of mirroring a 3D object from a first display to a second display may include: determining a relative orientation of the second display with respect to the first display; determining a relative size of the second display with respect to the first display; selecting a 3D mirroring mode; calculating a set of viewing parameters based on the relative orientation, relative size, and 3D mirroring mode; and transmitting a description of the 3D object and the set of viewing parameters to the second display.

[0209] In some embodiments, a method of mirroring a 3D object from a first display to a second display may include: receiving a 3D content description from the first display; receiving a set of viewing parameters from the first display; receiving 3D content based on the received 3D content description; and rendering a light field visualization at the second display based on the received 3D content and the set of viewing parameters.

[0210] In some embodiments, a method of mirroring a 3D object from a first display to a second display may include: receiving a 3D content description from the first display; receiving a set of viewing parameters from the first display; receiving 3D content based on the received 3D content description; tracking a position of the first display relative to the second display; and rendering a light field visualization at the second display based on the received 3D content, the set of viewing parameters, and the tracked position of the first display.

[0211] In some embodiments, an exemplary method may include: initially displaying a 3D object on a first display; determining a position and orientation of the first display relative to a second display; determining mirrored object orientation information based on the position and orientation of the first display relative to the second display; and sending a request to the second display to display the 3D object, wherein the request includes the mirrored object orientation information.

[0212] For some embodiments of the example method, the initially displayed 3D object may have a face facing the user, and the mirrored object orientation information may describe the orientation of the mirrored 3D object such that the same face of the mirrored 3D object faces the user.

[0213] An exemplary apparatus according to some embodiments may include a processor and a non-transitory computer-readable medium storing instructions operable to perform any of the methods listed above.

[0214] It should be noted that the various hardware elements of one or more embodiments in the embodiments are referred to as "modules" that implement (i.e., implement, execute, etc.) the various functions described herein in conjunction with the corresponding modules. As used herein, a module includes hardware that is considered by a technician in the relevant field to be suitable for a given specific implementation (e.g., one or more processors, one or more microprocessors, one or more microcontrollers, one or more microchips, one or more application-specific integrated circuits (ASICs), one or more field programmable gate arrays (FPGAs), one or more memory devices). Each of the modules may also include instructions that can be executed to implement one or more functions described as being implemented by the corresponding module, and it should be noted that these instructions may be in the form of hardware (i.e., hard-wired) or include hardware instructions, firmware instructions, software instructions, etc., and may be stored in any suitable one or more non-transitory computer-readable media (such as commonly referred to as RAM, ROM, etc.).

[0215] Although features and elements are described above in particular combinations, it will be understood by those of ordinary skill in the art that each feature or element may be used alone or in any combination with other features and elements. In addition, the methods described herein may be implemented in a computer program, software, or firmware incorporated in a computer-readable medium for execution by a computer or processor. Examples of computer-readable storage media include, but are not limited to, read-only memory (ROM), random access memory (RAM), registers, cache memory, semiconductor memory devices, magnetic media (such as built-in hard disks and removable disks), magneto-optical media, and optical media (such as CD-ROM disks and digital versatile disks (DVDs)). A processor associated with software may be used to implement a radio frequency transceiver for a WTRU, UE, terminal, base station, RNC, or any host computer.

Claims

1. A method comprising: receiving information identifying a virtual 3D object and first scaling information of the virtual 3D object, wherein the virtual 3D object is displayed to a user on a 2D display device using the first scaling information; determining a second scaling ratio based on the first scaling ratio information; as well as The 3D representation of the virtual 3D object is displayed on a 3D display device using the second scaling ratio. 2 . The method according to claim 1 , wherein the first scaling information comprises a display size of the virtual 3D object on the 2D display device and a size of the 2D display device.

3. The method of claim 1 , wherein determining the second scaling factor further comprises: The second scaling ratio is determined based on a size of the 3D display device.

4. The method according to claim 1, further comprising: receiving information identifying a distance between the user and the 2D display device; as well as The second scaling factor is determined based on the distance between the user and the 2D display device.

5. The method of claim 4, wherein the determining of the second scaling factor comprises: The second scaling factor is determined to be inversely proportional to a distance between the user and the 2D display device.

6. The method according to claim 1, further comprising: receiving information identifying a position of the user relative to the 3D display device; as well as The second scaling factor is determined based on the position of the user relative to the 3D display device.

7. The method of claim 6, wherein the determining of the second scaling factor comprises: The second scaling factor is determined to be proportional to the position of the user relative to the 3D display device.

8. The method according to claim 1, further comprising: detecting clipping of the 3D representation of the virtual 3D object; as well as The depth of the 3D representation of the virtual 3D object is adjusted to minimize the clipping.

9. The method according to claim 1, further comprising: receiving information identifying a position of the user relative to the 3D display device; as well as The second scaling is adjusted based on movement of the user derived from the received position of the user.

10. The method of claim 1, wherein the 3D display device is one of an autostereoscopic display or a light field display. The method of claim 1 , wherein the information identifying the virtual 3D object comprises a 3D model of the virtual 3D object or a link to the 3D model.

12. The method according to claim 1, further comprising: receiving information identifying first orientation information, wherein the first orientation information is used to display the virtual 3D object to the user on the 2D display device; receiving information identifying a position of the user relative to the 3D display device; determining a second orientation based on the first orientation information and based on the position of the user relative to the 3D display device; as well as The 3D representation of the virtual 3D object is displayed on the 3D display device using the second orientation.

13. The method according to claim 12, further comprising: receiving information identifying an orientation of the 2D display device; as well as The second orientation is determined based on the orientation of the 2D display device. The method of claim 12 , wherein the position of the user relative to the 3D display device is determined based on a position of the 2D display device.

15. A system comprising: at least one processor; as well as a memory storing instructions that, when executed by the at least one processor, cause the system to: receiving information identifying a virtual 3D object and first scaling information of the virtual 3D object, wherein the virtual 3D object is displayed to a user on a 2D display device using the first scaling information; determining a second scaling factor based on the first scaling factor information; and The 3D representation of the virtual 3D object is displayed on a 3D display device using the second scaling ratio. 16 . The system according to claim 15 , wherein the first scaling information comprises a display size of the virtual 3D object on the 2D display device and a size of the 2D display device.

17. The system of claim 15, wherein determining the second scaling factor further comprises: The second scaling ratio is determined based on a size of the 3D display device.

18. The system of claim 15, wherein the instructions further cause the system to: receiving information identifying a distance between the user and the 2D display device; and The second scaling factor is determined to be inversely proportional to a distance between the user and the 2D display device.

19. The system of claim 15, wherein the instructions further cause the system to: receiving information identifying a position of the user relative to the 3D display device; and The second scaling factor is determined to be proportional to the position of the user relative to the 3D display device.

20. The system of claim 15, wherein the instructions further cause the system to: receiving information identifying first orientation information, wherein the first orientation information is used to display the virtual 3D object to the user on the 2D display device; receiving information identifying a position of the user relative to the 3D display device; determining a second orientation based on the first orientation information and based on the position of the user relative to the 3D display device; and The 3D representation of the virtual 3D object is displayed on the 3D display device using the second orientation.

21. The system of claim 20, wherein the instructions further cause the system to: receiving information identifying an orientation of the 2D display device; and The second orientation is determined based on the orientation of the 2D display device.

22. The system of claim 20, wherein the position of the user relative to the 3D display device is determined based on a position of the 2D display device.

Images