Identify replacement 3D images for 2D images by ranking criteria

Abstract

A system and method for replacing a 2D image with an equivalent 3D image within a web page. The content of a 2D image displayed within a web page can be identified, and a 3D image can be identified as a possible replacement for the 2D image. The 3D images can be ranked based on multiple sets of ranking criteria. The 3D image with the highest ranking value can be selected based on the ranking of the 3D images. The selected 3D image can be integrated into the web page, thereby replacing the 2D image with the selected 3D image. In addition, user input for manipulating the 3D image within the web page can be received. The user input can include movement of a user's viewpoint relative to a display displaying the web page and / or detection of a light beam projected from one end of a user input device intersecting the 3D image.

Description

Technical Field

[0001] The present disclosure relates to the field of digital displays, and more particularly to methods of replacing two-dimensional images in a web page with substantially equivalent three-dimensional images. Background Art

[0002] Three-dimensional (3D) displays (actually simulated 3D, eg, via stereoscopic display (SD) technology) are increasingly being used for a variety of applications including, for example, remote viewing, video conferencing, video collaboration, and the like.

[0003] FIG1 illustrates a modern display chain according to a typical prior art embodiment, which comprises the following components:

[0004] 1. GPU - Graphics Processing Unit. This component resides on a personal computer, workstation, or functionally equivalent device and outputs a video level for each color or channel of the supported color model (e.g., for each of the three colors, typically red (R), green (G), and blue (B)) for each pixel on the display. Each of these numbers is typically an 8-bit number ranging from 0 to 255, but other ranges are possible.

[0005] 2. Scaler – This component takes as input the video levels for each pixel (e.g., for R, G, and B) output from the GPU, processes them in various ways, and then outputs (usually) the modified video levels for RGB, typically in the same 8-bit range of 0-255. This component can also scale the image from the input resolution to a different, rendered resolution supported by the display.

[0006] 3. Panel - This component is the display itself, usually a liquid crystal display (LCD), but other displays are possible, and takes as input the video levels for each pixel (e.g., for R, G, and B) output from the scaler and converts the video levels into voltages, which are then delivered to each pixel on the display. The panel itself may modify the video levels before converting them into voltages.

[0007] The video chain typically modifies video levels in two ways, specifically gamma correction and overdrive. Note that the above functions are typically implemented in a scaler, but are sometimes at least partially implemented in other devices or elements of the video chain, for example, in a GPU or display device (panel).

[0008] Time Sequential Stereoscopic Display

[0009] Unlike normal (i.e., monoscopic) displays, in stereoscopic displays, each video frame has two images: a right image and a left image. The right image can only be delivered to the right eye of the observer, and the left image can only be delivered to the left eye of the observer. In a time-sequential stereoscopic display, this separation of the left and right images is performed in time, i.e., the left and right images are presented sequentially and therefore contain some time-related element that separates the two images. There are two common architectures for stereoscopic displays.

[0010] The first architecture uses a device called a polarization switch (PS), which can be a distinct (i.e., separate) or integrated LC device or other technology switch, and is placed in front of an LCD panel (or any other type of imaging panel, such as an OLED (organic light emitting diode) panel, a plasma display, etc.) or any other pixelated panel display used in a time-sequential stereoscopic imaging system. Specifically, the PS switch can be placed between the display panel and the viewer, as shown in FIG2A . The purpose of the PS is to switch the light transmitted from the display panel between two orthogonal polarization states. For example, one of these states can be horizontal linear polarized light (i.e., the light can be in a horizontal linear polarization state), and the other state can be vertical linear polarized light (i.e., the light can be in a vertical linear polarization state); however, other options are also possible (e.g., left and right circular polarization states, etc.). The key feature that allows the PS to deliver the correct image to the correct eye of the viewer (i.e., left image to the left eye and right image to the right eye) is that the two polarization states are orthogonal to each other.

[0011] This system architecture allows the stereoscopic effect shown in prior art FIG. 2B to be achieved. As can be seen, the top of the figure shows a (display) panel that switches between a left image and a right image. As shown, in synchronization with the panel switching, the PS is switching the transmitted light between the left state and the right state. As described above, these two states emit two corresponding orthogonal polarization states. As further shown in FIG. 2B, the system includes a stereoscopic eyewear that is designed so that the left lens will only pass the left state polarization, and the right lens will only pass the right state polarization. In this way, the separation of the left and right images is achieved.

[0012] The second conventional system architecture uses stereo shutter glasses, which replace the PS and eyewear (or integrate the functions of the PS and eyewear). In such a system, each eye is covered by an optical shutter, which can be open or closed. Each of these shutters cycles between opening and closing in synchronization with the display panel, so that when the left image is displayed on the display, only the left eye shutter is open, and when the right image is displayed on the display, only the right eye shutter is open. In this way, the left view and the right view are alternately presented to the user's left eye and right eye, respectively. Figure 4As shown, alternating the left view and the right view to the user's left eye and right eye can create a visual sense of depth. Virtual objects can be displayed in this created 3D space, which exists both above the 3D stereo display panel (i.e., negative space) and below the stereo display panel (i.e., positive space).

[0013] Liquid Crystal Display (LCD)

[0014] Some advantages of liquid crystal displays (LCDs) include lighter weight, lower power consumption, and less radiation pollution. LCD monitors have been widely used in various portable information products (e.g., user equipment devices such as laptops, tablets, mobile phones, PDAs, etc.). Generally, in LCD monitors, when the arrangement of liquid crystal molecules changes, the incident light will produce different polarization or refraction effects. The liquid crystal molecules affect the transmission of the incident light, so the amount of light emitted from the liquid crystal molecules changes. LCD monitors use the characteristics of liquid crystal molecules to control the corresponding light transmittance and produce images of each different amount of red, blue, and green light.

[0015] A schematic diagram of a nematic liquid crystal phase 1000 is shown in FIG2C . Liquid crystal materials do not have a long-range ordering of the positions of their molecular centers of mass as in a crystal. However, liquid crystal materials have a long-range directional ordering of their molecules along the major axis (in the simplest case of so-called nematic liquid crystals), effectively aligning the molecules along a preferred direction, which is called the director of the liquid crystal. (See Figure 2C).

[0016] Liquid crystal molecules either have a permanent dipole moment or acquire an induced dipole moment when placed in an electric field. In both cases, in an electric field, the liquid crystal molecule 2000 is characterized by a certain dipole moment μ. This dipole can be aligned along the symmetry axis of the molecule (such a material is said to have a positive dielectric constant anisotropy) or it can be perpendicular to it (negative dielectric constant anisotropy). The separation of the charge in the molecule causes it to rotate in the electric field until it is aligned parallel or perpendicular to the applied field, depending on the sign of the dielectric anisotropy of the material. Figure 2D depicts this reorientation of a liquid crystal molecule with positive dielectric anisotropy.

[0017] Since all molecules in a liquid crystal phase reorient themselves simultaneously under the influence of an electric field, the symmetry axis (director) of the phase, typically the optical axis of a liquid crystal sample, can be controlled.

[0018] FIG2E illustrates the configuration of liquid crystal molecules 300 in a conventional twisted nematic liquid crystal based polarization rotator. The nematic liquid crystal is selected to have a positive dielectric anisotropy. The left side of the figure illustrates the voltage off (OFF), 90 degree rotation state. The right side of the figure illustrates the voltage on (ON), 0 degree rotation state.

[0019] Depending on the type of liquid crystal cell and the relative orientation of the optical axis of the liquid crystal cell and the transmission axis of the polarizer, the polarization rotator can operate in a normally white (NW) or normally black (NB) mode. These modes are controlled by the light transmission in the zero or low voltage state, i.e., the normally white mode corresponds to the maximum light transmission in the zero or low voltage state and the minimum transmission in the high voltage state; for the normally black mode, the opposite is true.

[0020] Twisted nematic polarization rotators are usually operated in normally white mode. In this case, a higher applied voltage improves the contrast ratio of the normally white mode due to a reduction in the residual retardation of the liquid crystal cell.

[0021] Other types of polarization rotators, such as the electrically controlled birefringence (ECB) mode, can operate in both normally white and normally black modes. Using additional optical elements in the 3D system (such as two orthogonal polarizers), the same polarization rotator can alternately operate in the two modes in each frame. Summary of the invention

[0022] Various embodiments of systems for implementing a method of replacing a two-dimensional (2D) image in a web page with a substantially equivalent three-dimensional (3D) image are disclosed herein. In some embodiments, the 3D image can be derived from a replacement 3D imagery. The replacement 3D imagery (or imagery) can be obtained from a website being accessed (e.g., hosting the web page) and / or from elsewhere on the Internet. In addition, various embodiments of systems for implementing a method for determining and / or selecting (e.g., based at least in part on one or more criteria) an optimal (e.g., substantially equivalent) 3D replacement image are disclosed herein.

[0023] In some embodiments, a system may include one or more displays and at least one processor coupled to (or in communication with) the one or more displays. In some embodiments, the one or more displays and at least one processor may be included in a head-mounted display system and / or communicate with the head-mounted display system. In some embodiments, the system may also include a stylus communicatively coupled to the at least one processor and a memory coupled to the at least one processor. The processor may be configured to execute or implement an embodiment of the technology disclosed herein. In addition, the method of implementing an embodiment of the technology disclosed herein may be implemented on a computer system or stored as a program instruction on a computer-readable storage medium. In some embodiments, the computer system may include a mobile device, such as a user equipment device (UE).

[0024] In some embodiments, a 2D image displayed within a web page can be identified. In addition, a 3D image having substantially equivalent content compared to the content of the 2D image can be identified. The 3D image can be integrated into the web page, thereby replacing the 2D image with the 3D image. In addition, at least one user input for manipulating the 3D image within the web page can be received. In some embodiments, at least one user input can include movement of the user's viewpoint (or point of view) relative to the display displaying the web page. In some embodiments, at least one user input can include detecting a light beam projected from one end of a user input device (and / or one end of the user input device) intersecting with the 3D image. In some embodiments, the intersection with the 3D image can be based at least in part on the tracking position and orientation of the user input device relative to the display displaying the web page.

[0025] In some embodiments, a 2D image displayed within a web page can be identified. In addition, a 3D image having substantially equivalent content compared to the content of the 2D image can be identified. The 2D image can be replaced by a flat 3D image, in which the z-scale value associated with the depth of the 3D image can be reduced to flatten the 3D image. In addition, at least one user input for manipulating the 3D image within the web page can be received. In some embodiments, at least one user input can include movement of the user's viewpoint (or observation point) relative to the display showing the web page. In some embodiments, at least one user input can include detecting a light beam projected from one end of a user input device (and / or one end of the user input device) intersecting with the 3D image. In some embodiments, the intersection with the 3D image can be based at least in part on the tracking position and orientation of the user input device relative to the display showing the web page.

[0026] In some embodiments, a 2D image displayed within a web page can be identified. In addition, a 3D image having substantially equivalent content compared to the content of the 2D image can be identified. The 3D image can be integrated into the web page, thereby replacing the 2D image with the 3D image. In addition, at least one user input for manipulating the 3D image within the web page can be received. In some embodiments, at least one user input can include movement of the user's viewpoint (or observation point) relative to the display displaying the web page. In some embodiments, at least one user input can include detecting a light beam projected from one end of a user input device (and / or one end of the user input device) intersecting with the 3D image. In some embodiments, the intersection with the 3D image can be based at least in part on the tracking position and orientation of the user input device relative to the display displaying the web page. In some embodiments, when receiving at least one user input for manipulating the 3D image within the web page, a determination can be made that the scale of the real world and / or the size of the 3D image are suitable within the range of the web browser displaying the web page, and the 3D image can be displayed with the scale and / or size of the real world.

[0027] In some embodiments, the content of a 2D image displayed within a web page can be identified. The 2D image can be identified based on one of a web page structure analysis or an identification of metadata associated with the 2D image. In addition, one or more three-dimensional (3D) images can be identified as possible replacements for the 2D image, for example, based on the content of the 2D image. One or more 3D images can be ranked, for example, based on one or more groups of ranking criteria. The 3D image with the highest ranking value can be selected and integrated into the web page, thereby replacing the 2D image with the 3D image. In addition, at least one user input for manipulating the 3D image within the web page can be received. In some embodiments, at least one user input can include movement of the user's viewpoint (or observation point) relative to a display displaying the web page. In some embodiments, at least one user input can include detecting a beam projected from one end of a user input device (and / or one end of the user input device) that intersects with the 3D image. In some embodiments, the intersection with the 3D image can be based at least in part on the tracking position and orientation of the user input device relative to the display displaying the web page.

[0028] In some embodiments, the content of a 2D image displayed within a web page can be identified, and one or more three-dimensional (3D) images can be identified as possible replacements for the 2D image, for example, based on the content of the 2D image. One or more 3D images can be ranked, for example, based on one or more sets of ranking criteria. A 3D image with a highest ranking value can be selected based on the ranking of one or more 3D images, for example, based on one or more sets of ranking criteria. An image of a 3D image that does not correspond to an image of a 2D image can be identified and modified so that the modified image of the 3D image corresponds to the image of the 2D image, thereby generating a modified 3D image. The modified 3D image can be integrated into the web page, thereby replacing the 2D image with the modified 3D image. In addition, at least one user input for manipulating the 3D image within the web page can be received. In some embodiments, at least one user input can include movement of a user's viewpoint (or observation point) relative to a display displaying the web page. In some embodiments, at least one user input can include detecting a beam projected from one end of a user input device (and / or one end of the user input device) that intersects with the modified 3D image. In some embodiments, the intersection with the modified 3D image may be based at least in part on a tracked position and orientation of the user input device relative to a display displaying the web page.

[0029] In some embodiments, the content of a 2D image displayed within a web page can be identified, and one or more three-dimensional (3D) images can be identified as possible replacements for the 2D image, for example, based on the content of the 2D image. One or more 3D images can be ranked, for example, based on one or more sets of ranking criteria. A 3D image with a highest ranking value can be selected based on the ranking of one or more 3D images, for example, based on one or more sets of ranking criteria. An image of a 3D image that does not correspond to an image of a 2D image can be identified and modified so that the modified image of the 3D image corresponds to the image of the 2D image, thereby generating a modified 3D image. The modified 3D image can be integrated into the web page, thereby replacing the 2D image with the modified 3D image. In addition, at least one user input for manipulating the 3D image within the web page can be received. In some embodiments, at least one user input can include movement of a user's viewpoint (or observation point) relative to a display displaying the web page. In some embodiments, at least one user input can include detecting a beam projected from one end of a user input device (and / or one end of the user input device) that intersects with the modified 3D image. In some embodiments, the intersection with the modified 3D image may be based at least in part on a tracked position and orientation of the user input device relative to a display displaying the web page.

[0030] In some embodiments, a 2D image displayed within a web page can be identified. In addition, a 3D image having substantially equivalent content compared to the content of the 2D image can be identified. The image of the 3D image that does not correspond to the image of the 2D image can be identified and modified so that the modified image of the 3D image corresponds to the image of the 2D image, thereby generating a modified 3D image. The modified 3D image can be integrated into the web page, thereby replacing the 2D image with the modified 3D image. In addition, at least one user input for manipulating the 3D image within the web page can be received. In some embodiments, at least one user input can include movement of the user's viewpoint (or observation point) relative to the display displaying the web page. In some embodiments, at least one user input can include detecting a beam projected from one end of a user input device (and / or one end of the user input device) that intersects with the modified 3D image. In some embodiments, the intersection with the modified 3D image can be based at least in part on the tracking position and orientation of the user input device relative to the display displaying the web page. BRIEF DESCRIPTION OF THE DRAWINGS

[0031] A better understanding of the present disclosure may be obtained when the following detailed description of the preferred embodiments is considered in conjunction with the following drawings, in which:

[0032] FIG. 1 illustrates a modern display chain according to an embodiment.

[0033] FIG. 2A illustrates an example of an architecture utilizing a polarization switch, according to an embodiment.

[0034] FIG. 2B illustrates an example of a stereoscopic effect (simulated 3D) using polarization switching between left and right views, according to an embodiment.

[0035] FIG. 2C illustrates an example of a schematic diagram of a nematic liquid crystal phase according to an embodiment.

[0036] FIG. 2D illustrates an example of a schematic diagram of switching liquid crystal molecules in an electric field according to an embodiment.

[0037] FIG. 2E illustrates an example of a schematic diagram of a configuration of liquid crystal molecules within a conventional twisted nematic liquid crystal-based polarization rotator according to an embodiment.

[0038] Figure 3A and Figure 3B An example of a 3D stereoscopic display system configured in accordance with some embodiments is illustrated.

[0039] Figure 4 An example block diagram of a user equipment device configured in accordance with some embodiments is illustrated.

[0040] Figure 5A and Figure 5B An example of a 3D head mounted stereoscopic display system configured in accordance with some embodiments is illustrated.

[0041] Figure 5C An example block diagram of a head mounted electronic device configured in accordance with some embodiments is illustrated.

[0042] Figure 6 An example of a user input device in accordance with some embodiments is illustrated.

[0043] Figure 7 Illustrated is a block diagram of an example of a process for identifying and replacing a 2D image with an equivalent 3D image in accordance with some embodiments.

[0044] Fig. 8A -D illustrates an example of one possible manipulation of a 3D image according to some embodiments.

[0045] Fig. 9 A block diagram illustrating an example of a process for flattening a 3D image when integrated into a web page to replace a 2D image in accordance with some embodiments.

[0046] Fig. 10A -C illustrates an example of another possible manipulation of a 3D image according to some embodiments.

[0047] Fig.11 A block diagram illustrating an example of a process for selecting an appropriate 3D image to replace a 2D image displayed on a web page in accordance with some embodiments.

[0048] Fig.12 A block diagram illustrating an example of a process for improving the suitability of 3D images to replace 2D images displayed on a web page in accordance with some embodiments.

[0049] While the present disclosure is susceptible to various modifications and alternative forms, specific embodiments thereof are shown by way of example in the drawings and described in detail herein. However, it should be understood that the drawings and their detailed description are not intended to limit the present disclosure to the particular forms disclosed, but on the contrary, are intended to cover all modifications, equivalents, and alternative forms falling within the spirit and scope of the present disclosure as defined by the appended claims. DETAILED DESCRIPTION

[0050] the term

[0051] The following is a glossary of terms used in this application:

[0052] Storage medium - any of various types of memory devices or storage devices. The term "storage medium" is intended to include: installation media, such as CD-ROM, floppy disk or tape device; computer system memory or random access memory, such as DRAM, DDR RAM, SRAM, EDO RAM, Rambus RAM, EEPROM, etc.; non-volatile memory, such as flash memory, magnetic media (e.g., hard drive) or optical storage device; registers or other similar types of memory elements, etc. Storage media may include other types of memory and combinations thereof. In addition, the storage medium may be located in a first computer in which the program is executed, or may be located in a different second computer connected to the first computer via a network such as the Internet. In the latter case, the second computer may provide program instructions to the first computer for execution. The term "storage medium" may include two or more storage media, which may reside in different locations, for example, in different computers connected via a network.

[0053] Carrier Medium—storage media as described above, and physical transmission media such as a bus, network, and / or other physical transmission media that convey signals such as electrical, electromagnetic or digital signals.

[0054] Computer system - any of various types of computing or processing systems, including personal computer systems (PCs), mainframe computer systems, workstations, network appliances, Internet appliances, personal digital assistants (PDAs), smart phones, television systems, grid computing systems, tablet computers, or other devices or combinations of devices. In general, the term "computer system" can be broadly defined to encompass any device (or combination of devices) having at least one processor that executes instructions from a storage medium.

[0055] Graphics Processing Unit - refers to a component that may reside on a personal computer, workstation, server, graphics server, or equivalent device and outputs a video level for each color or channel of a supported color model (e.g., for each of the three colors, typically red (R), green (G), and blue (B)) for each pixel on the display. Each of these numbers is typically an 8-bit number in the range of 0 to 255, but other ranges are possible.

[0056] Mobile device (or mobile station) - any of various types of computer system devices that are mobile or portable and perform wireless communications using WLAN communications. Examples of mobile devices include mobile phones or smart phones (e.g., iPhone TM , based on Android TM phones), and tablet computers such as iPads TM , Samsung GalaxyTM etc. Various other types of devices would fall into this category if they included Wi-Fi or both cellular and Wi-Fi communication capabilities, such as laptop computers (e.g., MacBooks TM ), portable gaming devices (e.g., Nintendo DS TM , PlayStation Portable TM 、Gameboy Advance TM , iPhone TM ), portable Internet devices and other handheld devices, as well as wearable devices such as smart watches, smart glasses, headphones, pendants, earpieces, etc. In general, the term "mobile device" can be broadly defined to cover any electronic, computing and / or telecommunication device (or combination of devices) that is easily transportable by a user and is capable of wireless communication using WLAN or Wi-Fi.

[0057] Wireless device (or wireless station) - Any of various types of computer system devices that perform wireless communications using WLAN communications. As used herein, the term "wireless device" may refer to a mobile device as defined above, or to a fixed device, such as a fixed wireless client or a wireless base station. For example, a wireless device may be any type of wireless station of an 802.11 system, such as an access point (AP) or a client station (STA or UE). Other examples include televisions, media players (e.g., AppleTV TM ,Roku TM , Amazon Fire TV TM , Google Chromecast TM etc.), refrigerators, washing machines, thermostats, etc.

[0058] WLAN - The term "WLAN" has the full scope of its ordinary meaning and includes at least a wireless communication network or RAT that is served by WLAN access points and provides connectivity to the Internet through these access points. Most modern WLANs are based on the IEEE 802.11 standard and are marketed under the name "Wi-Fi". WLAN networks are distinct from cellular networks.

[0059] Processing element (or functional unit) - refers to various implementations of digital circuitry that performs a function in a computer system. Additionally, a processing element may refer to various implementations of analog or mixed-signal (a combination of analog and digital) circuitry that performs a function (or functions) in a computer or computer system. Processing elements include, for example, circuits such as integrated circuits (ICs), ASICs (application specific integrated circuits), portions or circuits of individual processor cores, entire processor cores, individual processors, programmable hardware devices such as field programmable gate arrays (FPGAs), and / or larger portions of systems containing multiple processors.

[0060] Coupling Zone – refers to the physical volume within which a user of a 3D stereoscopic display can view 3D content within the natural depth of field of the human eye. Figure 4 As shown in FIG. 1 , when a person sees an object in the physical world, the person's eyes converge on the object, or fixate (aim alone) on the object. In addition, when both eyes converge on the object, the lens of each eye also focuses on the object through accommodation (monovision). In this sense, both eyes focus and converge on the object, so focus and convergence are "coupled".

[0061] Disparity – refers to the difference between the left-eye image and the right-eye image of a 3D stereoscopic display. Disparity can be described in at least two ways. First, for a display device, i.e., a 3D stereoscopic display, disparity can be described by the number of pixels separated between corresponding positions of a displayed or rendered image or content (in other words, the number of pixels separated between the left-eye and right-eye images or content). Alternatively, or in addition, for a user's observation point, disparity can be described by the degree of angular separation between corresponding positions in a displayed or rendered image or content (i.e., the angular separation between the left-eye and right-eye images or content).

[0062] Projection – refers to the display of 3D objects or content on a two-dimensional (2D) display. Thus, projection can be described as a mathematical function applied to an object within a virtual 3D scene to determine the object's virtual position within a 3D space that can be defined by the dimensions of the 3D stereoscopic display and the user's point of view.

[0063] Viewpoint - This term has the full range of its ordinary meaning in the computer graphics / camera field and specifies a position and / or orientation. For example, the term "viewpoint" may refer to a single point of view (e.g., for a single eye) or a pair of viewpoints (e.g., for a pair of eyes). Thus, a viewpoint may refer to the point of view of a single eye, or may refer to two viewpoints of a pair of eyes. "Single viewpoint" may specify that the viewpoint refers to only a single viewpoint, and "paired viewpoints" or "stereoscopic viewpoints" may specify that the viewpoint refers to two viewpoints (rather than one viewpoint). In the case where the viewpoint is the user's viewpoint, the viewpoint may be referred to as an eyepoint (see below) or a "physical viewpoint". The term "virtual viewpoint" refers to a viewpoint from within a virtual representation or 3D scene. Viewpoint is synonymous with "point of view" (POV) (see definition of POV below).

[0064] Eyepoint – The physical location (and / or orientation) of an eye or pair of eyes. The viewpoints above may correspond to the eyepoints of a person. For example, the eyepoints of a person have corresponding viewpoints.

[0065] Point of View (POV)-refers to or specifies a position and orientation. For example, POV can be a viewpoint or eye point of a typical user, but can also be a viewpoint of an optical device such as a camera. POV is generally a component that captures the relationship between two or more 6-DOF objects. In a typical application of the present technology, the user's eyes or head (view) are positioned to a display device (e.g., a monitor screen) in any X, Y, Z position and / or pitch, yaw, roll orientation, and the display device can have its own position in any X, Y, Z position and / or pitch, yaw, roll orientation. In this example, POV can be defined as the position / orientation of the user's field of view relative to the positioning / orientation of the display device. POV determination can be identified by a capture system. In a typical application of the present technology, one or more tracking devices are attached to the display device so that the controller knows what the tracking system is tracking in the context of the display device, i.e., the tracking system attached to the display device can be programmed to understand the position / orientation of the display device and any potential changes in the position / orientation of the display device.

[0066] The tracking system (which can, among other things, identify and track the user's field of view) can identify the position / orientation of the user's field of view, and this information can then be correlated with the tracking system's identification of the position / orientation of the viewing device (again, relative to the display device).

[0067] Vertical Perspective - The perspective effect rendered from a viewpoint that is substantially perpendicular to the display surface. "Substantially perpendicular" refers to 90 degrees or variations thereof, such as 89 or 91 degrees, 85-95 degrees, or any variation that does not cause noticeable distortion of the rendered scene. The vertical perspective can be a central perspective, e.g., having a single (and central) vanishing point. As used herein, vertical perspective can apply to a single image or a stereoscopic image. When used for a stereoscopic image (e.g., presenting a stereoscopic image according to a vertical perspective), each image of the stereoscopic image can be presented according to the vertical perspective, but using a different single viewpoint.

[0068] Horizontal or oblique viewing angle - a viewing angle effect rendered from a viewpoint that is not perpendicular to the display surface. More particularly, the term "horizontal viewing angle" may generally refer to a viewing angle effect rendered using a rendering plane that is angled at substantially 45 degrees relative to a corresponding viewpoint. Rendering may be intended for a display that may be positioned horizontally (e.g., parallel to a table surface or floor) relative to a standing viewpoint. "Substantially 45 degrees" may refer to 45 degrees or variations thereof, such as 44 and 46 degrees, 40-50 degrees, or any variation that may cause minimal distortion of the rendered scene. As used herein, a horizontal viewing angle may be applied to a single image or a stereoscopic image. When used for a stereoscopic image (e.g., a stereoscopic image is presented according to a horizontal viewing angle), each image of the stereoscopic image may be presented according to a horizontal viewing angle, but using a different single viewpoint.

[0069] Another concept of horizontal perspective, as commonly used in embodiments of the present technology, involves projecting the intended rendered graphics onto the viewing device. With the POV determined, the horizontal perspective engine can identify the correct graphics frustum in 3D space, taking into account the position and orientation of the viewing device when defining the rendering plane of the frustum and the position and orientation of the user's field of view to define the camera point of the frustum relative to the rendering plane. The resulting projection is then rendered onto the viewing device as the user would see it.

[0070] Position – The location or coordinates of an object (virtual or real). For example, position may include x, y, and z (i.e., position) coordinates within a defined space. Position may be relative or absolute, as desired. For example, when defining the orientation of a viewpoint, position may also include yaw, pitch, and roll information. In other words, position is defined broadly so as to encompass information about both position and orientation.

[0071] Stylus - a peripheral device or element such as a handheld device, handheld pen device, handheld pointing device, hand, finger, glove, or any object used to directly interact with a rendered virtual object as in a stereoscopically rendered virtual projected object.

[0072] Similar - as used herein with respect to geometric shapes, refers to a geometric term indicating that objects have the same shape, or that one object has the same shape as the mirror image of another object. In other words, if one object can be obtained from another object by uniformly scaling (enlarging or reducing) the object, then the objects are considered similar. In addition, the term similar or similar objects means that either object can be rescaled, repositioned, and reflected so as to coincide with another object. Thus, for example, if a first object is geometrically similar to a second object, i.e., has the same shape but may have different sizes, then either object can be uniformly scaled to obtain the geometric size and shape of the other object. Thus, the first object can be uniformly scaled to obtain the second object, or the second object can be uniformly scaled to obtain the first object. Note that the definition of similar refers only to the use of the word in the context of geometric shapes, and retains the ordinary meaning of the word in other contexts (e.g., system A is similar to system B means that system A is similar to system B, not the same as system B).

[0073] Approximately - refers to a value that is correct or exact within a certain specified tolerance. For example, approximately may refer to a value that is within 1% to 10% of an exact value (or desired value). However, it should be noted that the actual threshold value (or tolerance) may depend on the application. For example, in one embodiment, "approximately" may mean within 0.1% of a certain specified value or desired value, while in various other embodiments, the threshold value may be, for example, 2%, 3%, 5%, etc., as needed or as required by a particular application.

[0074] Close – Being near; for example, close can mean within a certain distance, or within a certain fraction of a distance. Note that the actual threshold for closeness is often application dependent. Thus, in various applications, close can mean within 1 mm, 1 inch, 1 foot, 1 meter, 1 mile, etc. of a reference point or object, or can mean within 1%, 2%, 5%, 10%, etc. of a reference distance from a reference point or object.

[0075] Substantially – refers to an approximate term. Similar to the term “approximately”, substantially refers to a certain tolerable range. Thus, if portion A is substantially horizontal, then portion A may be horizontal (90 degrees from vertical) or may be within a certain horizontal tolerance limit. For example, in one application, a range of 89-91 degrees from vertical may be tolerable, while in another application, a range of 85-95 degrees from vertical may be tolerable. Additionally, the tolerable limit may be one-sided. Thus, using the example of “portion A is substantially horizontal”, portion A may be tolerable within a range of 60-90 degrees from vertical, but not greater than 90 degrees from vertical. Alternatively, portion A may be tolerable within a range of 90-120 degrees from vertical, but not less than 90 degrees from vertical. Thus, the tolerable limit, and therefore the approximate value referenced by the use of the term substantially, may be what is desired or required for a particular application.

[0076] Equivalent - refers to an object that is equal or corresponds to another object in terms of value, measurement, function, meaning, effect, significance, appearance, etc. For example, if the imagery within the first image corresponds to the imagery within the second image, then the first image may be equivalent to the second image. In addition, if the imagery within the first image at least partially (e.g., within some tolerable range and / or limit) corresponds to the imagery within the second image, then the first image may be substantially equivalent to the second image.

[0077] Concurrency – refers to parallel operation or execution, where tasks, processes, or programs are executed in an at least partially overlapping manner. For example, concurrency can be achieved using "strong" or strict parallelism, where tasks are executed (at least partially) in parallel on various computing elements, or using "weak parallelism," where tasks are executed in an interleaved manner (e.g., by time multiplexing of threads of execution).

[0078] Automatic – refers to an action or operation performed by a computer system (e.g., software executed by a computer system) or a device (e.g., a circuit system, a programmable hardware element, an ASIC, etc.) without a user directly specifying or executing the action or operation. Thus, the term “automatic” is in contrast to an operation that is manually performed or specified by a user (where the user provides input to directly perform the operation). An automatic process may be initiated by input provided by a user, but the subsequent actions performed “automatically” are not specified by the user, i.e., are not performed “manually” (where the user specifies each action to be performed). For example, a user filling out an electronic form by selecting each field and providing input specifying information (e.g., by typing in information, selecting checkboxes, selecting radio buttons, etc.) is manually filling out a form, even though the computer system must update the form in response to the user action. A form may be automatically filled out by a computer system, where the computer system (e.g., software executing on the computer system) analyzes the fields of the form and fills out the form without any user input specifying the answers to the fields. As described above, a user may invoke automatic filling of a form but not participate in the actual filling out of the form (e.g., the user does not manually specify the answers to the fields, but rather they are automatically completed). This specification provides various examples of operations that are automatically performed in response to actions taken by a user.

[0079] Comprise - This term is open ended and means "comprising". As used in the appended claims, this term does not exclude additional elements, structures, or steps. Consider a claim stating: "A system including a display ...", such a claim does not exclude the system from including additional components (e.g., voltage source, light source, etc.).

[0080] Configured to - Various units, circuits, or other components may be described or stated as being "configured to" perform one or more tasks. In such contexts, "configured to" is used to represent structure by indicating that the unit / circuit / component includes structure (e.g., circuitry) that performs those one or more tasks during operation. Thus, even if the specified unit / circuit / component is not currently operational (e.g., not turned on), the unit / circuit / component can be said to be configured to perform a task. Units / circuits / components used with the "configured to" language include hardware—e.g., circuits, memory storing program instructions executable to implement an operation, etc. Stating that a unit / circuit / component is "configured to" perform one or more tasks is expressly intended to indicate that the unit / circuit / component includes structure (e.g., circuitry) that performs those one or more tasks during operation. No Reference is made to 35 USC §112(f) for this unit / circuit / component. Furthermore, "configured to" may include a general structure (e.g., general circuitry) manipulated by software and / or firmware (e.g., an FPGA or general purpose processor executing software) to operate in a manner capable of performing the task(s) in question.

[0081] First, second, etc. - These terms are used as labels for the nouns that precede them and do not imply any type of ordering (e.g., spatial, temporal, logical, etc.). For example, in a system with multiple tracking sensors (e.g., cameras), the terms "first" and "second" sensors can be used to refer to any two sensors. In other words, "first" and "second" sensors are not limited to logical sensors 0 and 1.

[0082] Based on - This term is used to describe one or more factors that influence a determination. The term does not exclude additional factors that may influence the determination. That is, the determination may be based solely on those factors or at least in part on those factors. Consider the phrase "determining A based on B." While B may be a factor that influences the determination of A, such a phrase does not exclude that the determination of A is also based on C. In other cases, A may be determined based solely on B.

[0083] This description may include references to "one embodiment" or "an embodiment." The appearances of the phrases "in one embodiment" or "in an embodiment" do not necessarily refer to the same embodiment. The particular features, structures, or characteristics may be combined in any suitable manner consistent with the present disclosure.

[0084] Figure 3-6 Exemplary system

[0085] Figure 3A and Figure 3B An exemplary system configured to implement various embodiments of the techniques described below is illustrated.

[0086] exist Figure 3A In an exemplary embodiment of the present invention, computer system 100A may include chassis 110A, display 150A and display 150B (which may be collectively referred to as display 150 or "one or more displays" 150), keyboard 120, mouse 125, user input device 130, eyewear 140, at least two cameras 160, and bracket 170. Note that in some embodiments, two displays 150A and 150B may not be used; instead, for example, a single display 150 may be used. In various embodiments, at least one of displays 150A and 150B may be a stereoscopic display. For example, in one embodiment, both displays 150A and 150B may be stereoscopic displays. Alternatively, in other embodiments, a single display 150 may be a stereoscopic display. Note that a stereoscopic display may also be configured to display two-dimensional (2D) objects and may be configured to operate in a 2D mode.

[0087] Chassis 110A may include various computer components, such as a processor, at least one storage medium (e.g., RAM, ROM, hard drive, etc.), a graphics circuit system, an audio circuit system, and other circuit systems for performing computer tasks, such as those described herein. At least one storage medium may store one or more computer programs or software components according to various embodiments of the present invention. For example, a storage medium may store one or more graphics engines that are executable to perform some of the techniques described herein. In some embodiments, the graphics engine may be implemented on or by a functional unit or processing element. As used herein, and as indicated in the terminology section above, the term functional unit or processing element refers to any of a variety of elements or combinations of elements configured to process instructions and / or data. Processing elements include, for example, circuits, such as ASICs (application specific integrated circuits), portions or circuits of individual processor cores, entire processor cores, individual processors, programmable hardware devices such as field programmable gate arrays (FPGAs), and / or larger portions of a system comprising multiple processors, and any combination thereof.

[0088] The storage medium (which may include two or more storage media) may also store data (and / or program instructions) representing a virtual space (e.g., implementing or specifying a computer model) that may be used to project a 3D scene (such as scene 180) of the virtual space via (one or more) displays 150. In addition, as described herein, the storage medium may store software executable to perform three-dimensional space tracking (e.g., user field of view tracking, user control tracking, etc.), content processing, or other features. For example, a computer system may include a tracking system that may track one or more of a user's head, a user's hand, or a stylus. In addition, the storage medium may store operating system software and other software for the operation of the computer system. Various embodiments also include receiving or storing instructions and / or data implemented according to the foregoing description on a carrier medium.

[0089] As described above, computer system 100A can be configured to display a three-dimensional (3D) scene (e.g., via a stereo image) or 3D content, such as scene 180, using display 150A and / or display 150B. Computer system 100A can also be configured to display a "view" of a 3D scene using display 150A, display 150B, and / or another display, as described in more detail below. A "view" of a 3D scene or content can refer to a displayed portion of a 3D scene from a viewpoint within the 3D scene. A viewpoint within a 3D scene can be referred to as a "virtual viewpoint." A view can be stereoscopic, for example, and can be displayed on a stereo display. Alternatively, a view can be monoscopic (not stereoscopic) and can be displayed on a monoscopic display or a stereoscopic display. Note that a monoscopic image or scene displayed on a stereoscopic display may appear the same as on a monoscopic display system.

[0090] It should be noted that Figure 3A The embodiments are merely exemplary, and other numbers of displays are also contemplated. For example, the computer system 100A may include only a single display or more than two displays, or the displays may be arranged in a manner different from that shown, such as goggles or other wearable eyewear or headgear as further described below with reference to FIG. 5 . In this particular embodiment, the display 150A is configured as a vertical display (which may be perpendicular or approximately perpendicular to the user's line of sight), and the display 150B is configured as a horizontal display (which may be parallel (or approximately parallel) or tilted to the user's line of sight). The vertical display 150A may be used (e.g., via instructions sent by a graphics engine executing in the chassis 110A) to provide images presented according to a vertical (or central) viewing angle, and the display 150B may be used (e.g., via instructions sent by a graphics engine executing in the chassis 110A) to provide images presented according to a horizontal viewing angle. A description of horizontal and vertical viewing angles is provided herein (e.g., see the terminology section above). Furthermore, although the displays 150 are shown as flat panel displays, in other embodiments they may be any type of device or system capable of displaying an image, such as a projection system. For example, the display(s) 150 may be or include a CRT (cathode ray tube) monitor, an LCD (liquid crystal display) monitor, or a front or rear projection screen or surface with multiple projectors, etc. The display(s) 150 may include a light emitting diode (LED) backlight or other type of backlight.

[0091] Either or both of the displays 150A and 150B may present (display) stereoscopic images for viewing by the user. By presenting stereoscopic images, the display(s) 150 may present a 3D scene to the user. The 3D scene may be viewed or referred to as an illusion or simulated 3D because the actual provided image is 2D, but the scene is conveyed in 3D via the stereo effect via the user's interpretation of the provided image. In order to properly view the stereoscopic images (one image per eye per image frame), the user may wear eyewear 140. The eyewear 140 may be any of anaglyph glasses, polarized glasses, shutter glasses, biconvex glasses, etc. In some embodiments, the display(s) 150 may be included (or incorporated) in the eyewear (or other wearable head-mounted device). In embodiments using anaglyph glasses, an image for a first eye is presented according to a first color (and the corresponding lens has a corresponding color filter), and an image for a second eye is projected according to a second color (and the corresponding lens has a corresponding color filter). For polarized glasses, an image is presented to each eye using orthogonal polarizations, and each lens of the eyewear has a corresponding orthogonal polarization for receiving the corresponding image. For shutter glasses, each lens is synchronized, for example, in an alternating manner, with respect to the left eye and right eye images provided by (one or more) displays 150. The display can provide both polarizations simultaneously or in an alternating manner (e.g., sequentially) as desired. Thus, the left eye can be allowed to see only the left eye image during the left eye image display time, and the right eye can be allowed to see only the right eye image during the right eye image display time. For biconvex glasses, the image is formed on a cylindrical lens element or a two-dimensional array of lens elements. Stereoscopic images can be provided via optical methods, where optical components such as prisms, (one or more) mirrors, (one or more) lenses, etc. are used to provide the left eye and right eye images only to the corresponding eyes. Large convex or concave lenses can also be used to receive two separately projected images for the user.

[0092] In one embodiment, eyewear 140 may be used as a positional input device for tracking a user's field of view (e.g., eyepoint or point of view (POV)) of a user viewing a 3D scene presented by system 100A. For example, eyewear 140 may provide information (e.g., positional information including orientation information, etc.) that may be used to determine the position of the user's point of view, for example, via triangulation. In some embodiments, the positional input device may use a light-sensitive detection system, for example, which may include an infrared detection system, to detect the position of the viewer's head to allow the viewer's head to move freely. Other embodiments of the input device(s) may use triangulation methods to detect the viewer's point of view position, such as one or more sensors (e.g., two cameras, such as a charge coupled device (CCD) or complementary metal oxide semiconductor (CMOS) camera) providing positional data suitable for head tracking. The viewer may manually operate the input device(s), such as a stylus, keyboard, mouse, trackball, joystick, etc., or a combination thereof, to specify or indicate the correct display of a horizontal viewing angle display image. However, any method for tracking the position of the user's head or point of view may be used as desired. Thus, the 3D scene can be rendered from the user's perspective (or observation point) so that the user can view the 3D scene with minimal distortion (e.g., because it is based on the user's observation point). Thus, the 3D scene can be rendered specifically for the user's observation point using a position input device.

[0093] The relationship between the position of display(s) 150 and the user's viewpoint can be used to map a portion of the virtual space to the physical space of system 100A. In essence, the physical space and components used can be mapped to the virtual model in order to accurately render the 3D scene of the virtual space.

[0094] One or more of the user input devices (e.g., keyboard 120, mouse 125, user input device 130, pointing device, user control device, user hand / fingers, etc.) can be used to interact with the presented 3D scene. For example, the user input device 130 (shown as a stylus) or simply the user's hand can be used to directly interact with virtual objects of the 3D scene (via the projected objects being viewed). This direct interaction may be with the negative space portion of the 3D scene. In some embodiments, at least a portion of the 3D scene can be presented in the negative space via stereo rendering (of the 3D scene), which is in front of at least one display or otherwise outside the display. In some embodiments, at least a portion of the 3D scene can appear as a hologram-like image above the surface of the display 150. For example, when the horizontal display 150B is used, the 3D scene can be seen hovering above the horizontal display. However, it should be noted that a portion of the 3D scene can also be presented to appear to be behind a display surface located in the positive space. Thus, negative space refers to space that a user can freely move into and interact with (e.g., where the user can place his hands (or more generally, user input device 130) in that space), as opposed to space that a user cannot freely move into and interact with (e.g., where the user cannot place his hands (or user input device 130) such as in a space below a display surface). Thus, negative space can be considered a "hands-on volume", as opposed to an "interior volume" (i.e., positive space), which may be below the surface of the display(s) and is therefore inaccessible. Thus, a user can interact with virtual objects in negative space because they are proximate to the user's own physical space. In other words, positive space is located behind (or below) the viewing surface, so the presented objects appear to be located inside (or behind) the physical viewing device. Thus, objects of a 3D scene presented in positive space do not share the same physical space as the user, and therefore, these objects cannot be directly and physically manipulated by hand or physically intersected by a handheld tool such as user input device 130. Rather, they may be manipulated indirectly, for example via a computer mouse, a joystick, a virtual representation of a hand, a hand tool or a stylus, or through a projection of a stylus (eg, a virtual laser or a virtual plane).

[0095] In some embodiments, the system 100A may include one or more sensors 160. One or more sensors 160 may be included in the tracking system. FIG. 3 illustrates an embodiment using four cameras 160. For example, two of the four cameras 160 may be used to sense the user's field of view (e.g., viewing point), while the other two cameras 160 may be used to sense the user input device (e.g., pointing device, stylus, hand, glove, etc.). Alternatively, less than four sensors (e.g., two sensors) may be used, each of which may simultaneously track the user (e.g., the user's head and / or the user's viewing point) and the user input device. The sensors 160 may be used to image the user of the system 100A, track the user's movement, or track the user's head or eyes and other intended functions. In one embodiment, the camera 160 may track the position and / or orientation of the user input device 130. The information about the position (including orientation) of the user input device 130 provided by the one or more sensors 160 may be used in combination with other position information of the system (e.g., accelerometers and / or gyroscopes within the stylus itself) to perform more accurate 3D tracking of the user input device 130. One or more sensors 160 may be spatially separated from one another and placed in a position to view a volume that encompasses the volume in which the user will view the stereoscopic image. The sensors 160 may also be sufficiently far from one another to provide line of sight for true three-axis triangulation determination. The system 100A may also include a cradle 170 to store the user input device 130. The cradle 170 may also be used to calibrate the orientation of the stylus to a known roll, pitch, and yaw, and thus may be in a fixed position relative to the camera 160.

[0096] In one embodiment, system 100A may be configured to be coupled to a network, such as a wide area network, via an input. The input may be configured to receive data (e.g., image data, video data, audio data, etc.) from a system similar to system 100A over the network. In other embodiments, the tracking system may include camera 160. Camera 160 may be configured to provide visual information about the user (e.g., so that the user's POV can be determined, such as the position (including orientation), or so that the position of the user's hand can be determined). However, it should be noted that any type of various tracking techniques or devices may be used as desired. Note that, as used herein, the user's POV refers to the perspective or POV from which the user optically views an object or image, i.e., the user's visual POV, and is therefore defined relative to the display device of the system. In some embodiments, the POV may be a 6-degree-of-freedom (6DOF) POV, e.g., three position coordinates and three orientation coordinates, but any POV may be used as desired, e.g., three position coordinates and two or three orientation coordinates, etc. As described above, the position coordinates may include position and orientation coordinates.

[0097] Note that in some embodiments, the tracking system may rely at least in part on components of rack 110A to determine the position or POV, for example, via determination of the position or POV by a processor or functional unit of rack 110A or one or more programs executed thereon, but in other embodiments, the tracking system may operate independently, for example, may have its own processor or functional unit.

[0098] In some embodiments, the system may include components that implement a perspective-based image capture system for capturing images of a target object at a location remote from the system. For example, the perspective-based image capture system may include an input configured to couple to a network to receive information about a point of view (POV) from a tracking system at a remote location. The information about the POV may indicate the location of the remote user. The perspective-based image capture system may also include another image capture system for capturing images of the target object. More specifically, the image capture system may be configured to capture one or more images from a first perspective based on information about the POV received by the input.

[0099] The user may be able to specify or otherwise manipulate a virtual viewpoint in the 3D scene presented by display(s) 150. As desired, a view of the 3D scene based on the virtual viewpoint may be presented through one or more or another of display(s) 150. The view of the 3D scene may be stereoscopic or monoscopic as desired.

[0100] A 3D scene generator (e.g., a content processing system) stored and executed in the rack 110A can be configured to dynamically change the display image provided by the (one or more) displays 150. More specifically, the 3D scene generator can update the displayed 3D scene based on changes in the user's field of view, user control (e.g., via manipulation of a user input device), etc. Such changes can be performed dynamically at runtime and can be performed in real time. The 3D scene generator can also track peripheral devices (e.g., user input devices 130 or eyewear 140) to ensure synchronization between the peripheral devices and the display image. The system can also include a calibration unit, process, and / or fiducial markers to ensure correct mapping of peripheral devices to the display image and correct mapping between the projected image and the virtual image stored in the memory of the rack 110A.

[0101] Thus, the system 100A can present a 3D scene with which a user can interact in real time. The system may include (one or more) real-time electronic displays 150 that can present or convey a perspective image in an open space, and a user input device 130 that can allow a user to interact with the 3D scene using a manually controlled or handheld tool. The system 100A may also include components for manipulating the displayed image in various ways, such as enlarging, scaling, rotating or moving, or even displaying a new image. However, as described above, in some embodiments, the system may facilitate such manipulation via the user's hands, for example without the need for a handheld tool.

[0102] Furthermore, while the system 100A is shown as including a horizontal display 150B because it simulates the visual experience of a user on horizontal ground, other viewing surfaces can provide similar 3D illusion experiences. For example, by projecting a horizontal perspective image onto a ceiling surface, a 3D scene can appear to be hanging from the ceiling, or by projecting a horizontal perspective image onto a vertical wall surface, a 3D scene can appear to be floating from the wall. More generally, any other variations of display orientation and perspective (or any other configuration of the system 100A) can be used as desired.

[0103] According to various embodiments of the present disclosure, the display 150 can display various types of information (e.g., multimedia data or text data) to be provided to the user. The display 150 can be configured to include a liquid crystal display (LCD), a light emitting diode (LED) display, an organic LED (OLED) display, a plasma cell display, an electronic ink array display, an electronic paper display, a flexible LCD, a flexible electrochromic display, or a flexible electrowetting display. The display 150 can be functionally connected to (one or more) elements of the electronic device. Moreover, the display 150 can be functionally connected to (one or more) electronic devices other than the electronic device.

[0104] exist Figure 3B In an exemplary embodiment of the present invention, computer system 100B may include housing 110B, which may include display 150, keyboard 120, trackpad or touchpad 135, and at least two cameras 160. Computer system 100B may also include user input device 130 and eyewear 140. Note that in some embodiments, computer system 100B may be a wireless or mobile station, such as wireless station 106 described further below. For example, computer system 100B may be or include in a mobile phone or smart phone (e.g., an iPhone-based TM 、Android TM phones), tablet computers (e.g., iPad TM 、Samsung GalaxyTM etc.), laptop computers (e.g., MacBook TM ), portable gaming devices (e.g., Nintendo DS TM , PlayStation Portable TM 、Gameboy Advance TM , iPhone TM ), portable Internet devices and / or other handheld devices. In various embodiments, at least one of the displays 150 may be a stereoscopic display. Note that a stereoscopic display may also be configured to display two-dimensional (2D) objects and may be configured to operate in a 2D mode.

[0105] Chassis 110B may include various computer components, such as a processor, at least one storage medium (e.g., RAM, ROM, hard drive, etc.), a graphics circuit system, an audio circuit system, and other circuit systems for performing computer tasks, such as those described herein. At least one storage medium may store one or more computer programs or software components according to various embodiments of the present invention. For example, a storage medium may store one or more graphics engines that are executable to perform some of the techniques described herein. In some embodiments, the graphics engine may be implemented on or by a functional unit or processing element. As used herein, and as indicated in the terminology section above, the term functional unit or processing element refers to any of a variety of elements or combinations of elements configured to process instructions and / or data. Processing elements include, for example, circuits, such as ASICs (application specific integrated circuits), portions or circuits of individual processor cores, entire processor cores, individual processors, programmable hardware devices such as field programmable gate arrays (FPGAs), and / or larger portions of a system comprising multiple processors, and any combination thereof.

[0106] The storage medium (which may include two or more storage media) may also store data (and / or program instructions) representing a virtual space (e.g., implementing or specifying a computer model) that may be used to project a 3D scene (such as scene 180) of the virtual space via (one or more) displays 150. In addition, as described herein, the storage medium may store software executable to perform three-dimensional space tracking (e.g., user field of view tracking, user control tracking, etc.), content processing, or other features. For example, a computer system may include a tracking system that may track one or more of a user's head, a user's hand, or a stylus. In addition, the storage medium may store operating system software and other software for the operation of the computer system. Various embodiments also include receiving or storing instructions and / or data implemented according to the foregoing description on a carrier medium.

[0107] As described above, computer system 100B (or more specifically, rack 110B) can be configured to display a three-dimensional (3D) scene (e.g., via a stereoscopic image) or 3D content, such as scene 180, using display 150. Computer system 100B can also be configured to display a "view" of a 3D scene using display 150. A "view" of a 3D scene or content can refer to a displayed portion of the 3D scene from a viewpoint within the 3D scene. A viewpoint within a 3D scene can be referred to as a "virtual viewpoint." The view can be stereoscopic, for example, and can be displayed on a stereoscopic display. Alternatively, the view can be monoscopic (not stereoscopic) and can be displayed on a monoscopic display or a stereoscopic display. Note that a monoscopic image or scene displayed on a stereoscopic display may appear the same as on a monoscopic display system.

[0108] In some embodiments, display 150 may present a 3D scene to the user. The 3D scene may be viewed or referred to as an illusion or simulated 3D because the actual provided image is 2D, but the scene is conveyed in 3D via a stereoscopic effect via the user's interpretation of the provided image. In order to properly view the stereoscopic images (one image per eye per image frame), the user may wear eyewear 140. Eyewear 140 may be any of anaglyph glasses, polarized glasses, shutter glasses, biconvex glasses, and the like.

[0109] In some embodiments, eyewear 140 may be used as a positional input device for tracking a user's field of view (e.g., eye point or point of view (POV)) of a user viewing a 3D scene presented by system 100B. For example, eyewear 140 may provide information (e.g., positional information including orientation information, etc.) that may be used to determine the position of the user's point of view, for example, via triangulation. In some embodiments, the positional input device may use a light-sensitive detection system, for example, which may include an infrared detection system, to detect the position of the viewer's head to allow the viewer's head to move freely. Other embodiments of the input device(s) may use a triangulation method to detect the viewer's point of view position, such as one or more sensors (e.g., two cameras, such as a charge coupled device (CCD) or complementary metal oxide semiconductor (CMOS) camera) providing positional data suitable for head tracking. The viewer may manually operate the input device(s), such as a stylus, keyboard, mouse, trackball, joystick, etc., or a combination thereof, to specify or indicate the correct display of a horizontal viewing angle display image. However, any method for tracking the position of the user's head or point of view may be used as desired. Thus, the 3D scene can be rendered from the user's perspective (or observation point) so that the user can view the 3D scene with minimal distortion (e.g., because it is based on the user's observation point). Thus, the 3D scene can be rendered specifically for the user's observation point using a position input device.

[0110] The relationship between the position of display 150 and the user's viewpoint can be used to map a portion of the virtual space to the physical space of system 100B. In essence, the physical space and components used can be mapped to the virtual model in order to accurately render the 3D scene of the virtual space.

[0111] One or more of the user input devices (e.g., keyboard 120, touchpad 135, user input device 130, pointing device, user control device, user hand / fingers, etc.) can be used to interact with the presented 3D scene. For example, user input device 130 (shown as a stylus) or simply the user's hand can be used to directly interact with virtual objects of the 3D scene (via the projected objects being viewed). This direct interaction may be with the negative space portion of the 3D scene. In some embodiments, at least a portion of the 3D scene can be presented in the negative space via stereo rendering (of the 3D scene), which is in front of at least one display or otherwise outside of the display. In some embodiments, at least a portion of the 3D scene can appear as a hologram-like image above the surface of display 150. However, it should be noted that a portion of the 3D scene can also be presented to appear to be behind a display surface located in positive space. Thus, negative space refers to space that a user can freely move into and interact with (e.g., where the user can place his hand (or more generally, user input device 130) in that space), as opposed to space that a user cannot freely move into and interact with (e.g., where the user cannot place his hand (or user input device 130) in that space (such as the space below the display surface)). Thus, negative space can be considered a "hands-on volume" as opposed to an "interior volume" (i.e., positive space), which may be below the surface of the display(s) and is therefore inaccessible. Thus, a user can interact with virtual objects in negative space because they are proximate to the user's own physical space. In other words, positive space is located behind (or below) the viewing surface, so the presented objects appear to be located inside (or behind) the physical viewing device. Thus, objects of a 3D scene presented in positive space do not share the same physical space as the user, and therefore, these objects cannot be directly and physically manipulated by hand or physically intersected by a handheld tool such as user input device 130. Rather, they may be manipulated indirectly, for example via a computer mouse, a joystick, a virtual representation of a hand, a hand tool or a stylus, or by a projection from a stylus (eg, a virtual laser or a virtual plane).

[0112] In some embodiments, system 100 may include one or more sensors 160. One or more sensors 160 may be included in a tracking system. Figure 3BAn embodiment using four cameras 160 is illustrated. For example, two of the four cameras 160 can be used to sense the user's field of view (e.g., viewing point), while the other two cameras 160 can be used to sense the user input device (e.g., pointing device, stylus, hand, glove, etc.). Alternatively, less than four sensors (e.g., two sensors) can be used, each of which can track the user (e.g., the user's head and / or the user's viewing point) and the user input device at the same time. The sensors 160 can be used to image the user of the system 100B, track the user's movement, or track the user's head or eyes and other intended functions. In one embodiment, the camera 160 can track the position and / or orientation of the user input device 130. The information about the position (including orientation) of the user input device 130 provided by one or more sensors 160 can be used in combination with other position information of the system (e.g., accelerometers and / or gyroscopes within the stylus itself) to perform more accurate 3D tracking of the user input device 130. One or more sensors 160 can be spatially separated from each other and placed in a position to view a volume covering the user in which the stereo image will be viewed. The sensors 160 may also be far enough apart from each other to provide line of sight for true three-axis triangulation determination.

[0113] In some embodiments, system 100B may be configured to be coupled to a network, such as a wide area network, via an input or interface (wired or wireless). The input may be configured to receive data (e.g., image data, video data, audio data, etc.) from a system similar to system 100A or 100B via a network. In other embodiments, the tracking system may include a camera 160. Camera 160 may be configured to provide visual information about the user (e.g., so that the user's POV can be determined, such as a position (including orientation), or so that the position of the user's hand can be determined). However, it should be noted that any type of various tracking techniques or devices may be used as desired. Note that, as used herein, a user's POV refers to the perspective or POV from which the user optically views an object or image, i.e., the user's visual POV, and is therefore defined relative to the display device of the system. In some embodiments, the POV may be a 6-degree-of-freedom (6DOF) POV, e.g., three position coordinates and three orientation coordinates, but any POV may be used as desired, e.g., three position coordinates and two or three orientation coordinates, etc. As described above, the position coordinates may include position and orientation coordinates.

[0114] Note that in some embodiments, the tracking system may rely at least in part on components of rack 110B to determine the position or POV, for example, via determination of the position or POV by a processor or functional unit of rack 110B or one or more programs executed thereon, but in other embodiments, the tracking system may operate independently, for example, may have its own processor or functional unit.

[0115] In some embodiments, the system may include components that implement a perspective-based image capture system for capturing images of a target object at a location remote from the system. For example, the perspective-based image capture system may include an input configured to couple to a network to receive information about a point of view (POV) from a tracking system at a remote location. The information about the POV may indicate the location of the remote user. The perspective-based image capture system may also include another image capture system for capturing images of the target object. More specifically, the image capture system may be configured to capture one or more images from a first perspective based on information about the POV received by the input.

[0116] The user may be able to specify or otherwise manipulate a virtual viewpoint in a 3D scene presented by display 150. As desired, a view of the 3D scene may be presented based on the virtual viewpoint through one or more or another of displays 150. As desired, the view of the 3D scene may be stereoscopic or monoscopic.

[0117] A 3D scene generator (e.g., a content processing system) stored and executed in the housing 110B can be configured to dynamically change the display image provided by the display 150. More specifically, the 3D scene generator can update the displayed 3D scene based on changes in the user's field of view, user control (e.g., via manipulation of a user input device), etc. Such changes can be performed dynamically at runtime and can be performed in real time. The 3D scene generator can also track peripheral devices (e.g., user input device 130 or eyewear 140) to ensure synchronization between the peripheral devices and the display image. The system can also include a calibration unit, process, and / or fiducial markers to ensure correct mapping of peripheral devices to the display image and correct mapping between the projected image and the virtual image stored in the memory of the housing 110B.

[0118] Thus, the system 100B can present a 3D scene with which the user can interact in real time. The system may include a real-time electronic display 150 that can present or convey a perspective image in an open space, and a user input device 130 that can allow the user to interact with the 3D scene using a manually controlled or handheld tool. The system 100B may also include components for manipulating the displayed image in various ways, such as magnifying, scaling, rotating or moving, or even displaying a new image. However, as described above, in some embodiments, the system may facilitate such manipulation via the user's hands, for example without the need for a handheld tool.

[0119] According to various embodiments of the present disclosure, the display 150 may display various types of information (e.g., multimedia data or text data) to be provided to the user. The display 150 may be configured to include a liquid crystal display (LCD), a light emitting diode (LED) display, an organic LED (OLED) display, a plasma cell display, an electronic ink array display, an electronic paper display, a flexible LCD, a flexible electrochromic display, or a flexible electrowetting display. The display 150 may be functionally connected to (one or more) elements of an electronic device. Moreover, the display 150 may be functionally connected to (one or more) electronic devices other than the electronic device. According to various embodiments of the present disclosure, the input module 240 may receive input for controlling, for example, the properties of a history screen. The input module 240 may receive input such as "reference screen settings". "Reference screen settings" may include an operation for storing information related to a screen in the storage module 210 in order to display a reference screen. The input module 240 may receive, for example, input for displaying a reference screen. The properties of the screen may include, for example, at least one of the position of the reference screen, the volume of the reference screen, the brightness of the screen, and the size of the screen. According to various embodiments of the present disclosure, if the input module 240 is included in the second electronic device, the input module 240 may not be provided in the electronic device.

[0120] Figure 4 An example simplified block diagram of a wireless station 106 is shown. According to an embodiment, the wireless station 106 may be a user equipment (UE) device, a mobile device, and / or a mobile station. The wireless station 106 may be similar to the one described above with reference to Figure 3A and Figure 3B The system described and the following reference Figure 5B and Figure 5CThe wireless station 106 may be used in conjunction with the systems described herein. For example, the wireless station 106 may be configured as an input device for any of the systems described (e.g., the wireless station 106 may be configured as a user input device). As another example, according to some embodiments, the wireless station 106 may be configured as a display for any of the systems described. Thus, the wireless station 106 may be configured to display stereoscopic images. In some embodiments, the wireless station 106 may be configured to communicate with the 3D system wirelessly (e.g., via a local area network such as Wi-Fi, Bluetooth, or Bluetooth low energy connection) or via a wired interface such as a universal serial bus interface, as well as other wired interfaces. In some embodiments, the wireless station 106 may be included in a computer system, such as the computer system 100B described above.

[0121] As shown, the wireless station 106 may include a system on a chip (SOC) 300, which may include parts for various purposes. The SOC 300 may be coupled to various other circuits of the wireless station 106. For example, the wireless station 106 may include various types of memory (e.g., including NAND flash memory 310), a connector interface (I / F) (or docking station) 320 (e.g., for coupling to a computer system, a docking station, a charging station, etc.), a display 360, a cellular communication circuit system 330 (such as, for LTE, GSM, etc.), and a short-range and medium-range wireless communication circuit system 329 (e.g., Bluetooth TM The wireless station 106 may also include one or more smart cards 310 incorporating SIM (Subscriber Identity Module) functionality, such as one or more UICC (Universal Integrated Circuit Card) cards 345. The cellular communication circuitry 330 may be coupled to one or more antennas, such as antennas 335 and 336 as shown. The short-range and medium-range wireless communication circuitry 329 may also be coupled to one or more antennas, such as antennas 337 and 338 as shown. Alternatively, the short-range and medium-range wireless communication circuitry 329 may be coupled to antennas 335 and 336 in addition to or in lieu of being coupled to antennas 337 and 338. The short-range and medium-range wireless communication circuitry 329 may include multiple receive chains and / or multiple transmit chains for receiving and / or transmitting multiple spatial streams, such as in a multiple-input multiple-output (MIMO) configuration.

[0122] As shown, SOC 300 may include processor(s) 302 that may execute program instructions for wireless station 106 and display circuitry 304 that may perform graphics processing and provide display signals to display 360. Processor(s) 302 may also be coupled to a memory management unit (MMU) 340 that may be configured to receive addresses from processor(s) 302 and translate those addresses to locations in memory (e.g., memory 306, read-only memory (ROM) 350, NAND flash memory 310) and / or to other circuits or devices, such as display circuitry 304, cellular communication circuitry 330, short-range wireless communication circuitry 329, connector interface (I / F) 320, and / or display 360. MMU 340 may be configured to perform memory protection and page table translation or setup. In some embodiments, MMU 340 may be included as part of processor(s) 302.

[0123] As described herein, the wireless station 106 may include hardware and software components for implementing the features described herein, for example, the wireless station 106 may form at least a portion of a 3D display system (such as the system 100 described above and / or the systems 500A and 5B described below). For example, the processor 302 of the wireless station 106 may be configured to implement some or all of the features described herein, for example, by executing program instructions stored on a storage medium (e.g., a non-transitory computer-readable storage medium). Alternatively (or additionally), the processor 302 may be configured as a programmable hardware element, such as an FPGA (field programmable gate array), or as an ASIC (application-specific integrated circuit). Alternatively (or additionally), the processor 302 of the UE 106 in combination with one or more of the other components 300, 304, 306, 310, 320, 330, 335, 340, 345, 350, 360 may be configured to implement some or all of the features described herein.

[0124] Additionally, as described herein, processor 302 may include one or more processing elements. Thus, processor 302 may include one or more integrated circuits (ICs) configured to perform the functions of processor 302. Additionally, each integrated circuit may include a circuit system (e.g., a first circuit system, a second circuit system, etc.) configured to perform the functions of processor(s) 302.

[0125] refer to Figure 5A, the head-mounted electronic device 500A may include a main body 502A and a cover 504. The main body 502 may include lenses 508 and 510, and a control device 514. In addition, the electronic device 500A may include a support 506A, which may be configured to support the electronic device 500A on the user's head. The lenses 508 and 510 may be positioned to correspond to the user's eyes. The user can view the screen on the display through the lenses 508 and 510. The display may be coupled or connected to the electronic device 500. In some embodiments, the display may be included on (or in) the cover 504, and the cover 504 may be configured to be coupled to the main body 502A. In some embodiments, the electronic device 500B may include a display, such as the one described above with reference to FIG. Figure 4 Thus, cover 504 can be communicatively coupled to body 502A (e.g., to couple the display of cover 504 to a processor of electronic device 500) and mechanically coupled (e.g., attached to) body 502. In some embodiments, the communicative coupling between body 502A and cover 504 can be wired and / or wireless.

[0126] In some embodiments, the control device 514 may be located on a side surface of the body 502A. The control device 514 may be used for user input for controlling the head mounted electronic device 500A. For example, the control device 514 may include a touch panel, a button, a scroll wheel key, and / or a touch pad. The touch panel may receive a user's touch input. The touch input may be a direct touch input to the touch panel, or a hover input near the touch panel.

[0127] Go to Figure 5B , the head-mounted electronic device 500B may include a main body 502B and a support 506B. The main body 502B may be configured to be coupled to a wireless station, and the display of the electronic device 500B may be a display of a wireless station (such as the wireless station 106), and the wireless station may be coupled or connected to (e.g., may be detachably mounted to) the electronic device 500B. In other words, the electronic device 500B may be configured so that the wireless station may be non-permanently coupled to the electronic device 500B and may be movable to the electronic device 500B without destructive measures. Therefore, the electronic device 500B may be coupled and decoupled from the wireless station (e.g., non-destructively decoupled from it) without changing the functionality of the wireless station or the electronic device 500B.

[0128] Go to Figure 5C , Figure 5CAn example simplified block diagram of a head-mounted electronic device 500C is illustrated. Depending on the embodiment, the electronic device 500C may include a display (e.g., such as the electronic device 500A), or may be configured to be coupled to a wireless station (e.g., such as the electronic device 500B). Note that the electronic devices 500A and 500B described above may include at least some of the features described with reference to the electronic device 500C.

[0129] As shown, the electronic device 500C may include a system on chip (SOC) 506, which may include parts for various purposes. The SOC 506 may be coupled to various other circuits of the electronic device 500C. For example, the electronic device 500C may include various types of memory (e.g., including NAND flash memory 510), a connector interface (I / F) (or docking station) 520 (e.g., for coupling to a computer system, a docking station, a charging station, an external display, etc.), a display 560 (note that in some embodiments, the electronic device 500C may not include the display 560) and a short-range and medium-range wireless communication circuit system 529 (e.g., Bluetooth TM The short-range wireless communication circuitry 529 may also be coupled to one or more antennas, such as antennas 537 and 538 as shown. The short-range wireless communication circuitry 529 may include multiple receive chains and / or multiple transmit chains for receiving and / or transmitting multiple spatial streams, such as in a multiple-input multiple-output (MIMO) configuration.

[0130] As shown, SOC 506 may include processor(s) 502 that may execute program instructions for electronic device 500C and display circuitry 504 that may perform graphics processing and provide display signals to display 560 (and / or docking station 520). Processor(s) 502 may also be coupled to a memory management unit (MMU) 540 that may be configured to receive addresses from processor(s) 502 and translate those addresses to locations in memory (e.g., memory 506, read-only memory (ROM) 550, NAND flash memory 510) and / or to other circuits or devices, such as display circuitry 504, short-range wireless communication circuitry 529, connector interface (I / F) 520, and / or display 560. MMU 540 may be configured to perform memory protection and page table translation or setup. In some embodiments, MMU 540 may be included as part of processor(s) 502.

[0131] In some embodiments, electronic device 500C (and / or electronic devices such as electronic devices 500A or 500B) can communicate with a user input device such as the above-mentioned user input device 130. In some embodiments, the electronic device can receive user input via user input device 130 as described above.

[0132] In addition, in some embodiments, the electronic device 500C may include one or more position sensors, such as an accelerometer, a gyroscope sensor, a geomagnetic sensor, a magnetic sensor, a proximity sensor, a posture sensor, a grip sensor, and / or a biometric sensor. In some embodiments, the electronic device may use one or more position sensors to obtain information to determine the movement of a user wearing the electronic device and / or whether the user is wearing or removing the electronic device 500C. At least one processor may control the execution of (one or more) functions or (one or more) operations corresponding to the input received by the control device (e.g., the control device 514 and / or the user input device 130) in response to the received input.

[0133] As described herein, the electronic device 500C may include hardware and software components for implementing the features described herein, for example, the electronic device 500C may form at least a portion of a 3D display system (such as the system 100 described above and / or the systems 500A and 5B described below). For example, the processor 502 of the electronic device 500C may be configured to implement some or all of the features described herein, for example, by executing program instructions stored on a storage medium (e.g., a non-transitory computer-readable storage medium). Alternatively (or additionally), the processor 502 may be configured as a programmable hardware element, such as an FPGA (field programmable gate array), or as an ASIC (application-specific integrated circuit). Alternatively (or additionally), the processor 502 of the UE 106 may be configured to implement some or all of the features described herein in combination with one or more of the other components 500, 504, 506, 510, 520, 535, 550, 560.

[0134] In some embodiments, the electronic device 500C may include or communicate with one or more external cameras. For example, the electronic device 500C may include (or communicate with) one or more cameras (or camera arrays) that may be configured to capture images of the user's physical location.

[0135] Additionally, as described herein, processor 502 may include one or more processing elements. Thus, processor 502 may include one or more integrated circuits (ICs) configured to perform the functions of processor 502. Additionally, each integrated circuit may include a circuit system (e.g., a first circuit system, a second circuit system, etc.) configured to perform the functions of processor(s) 502.

[0136] Figure 6 An example of a user input device according to some embodiments is shown. As shown, the user input device 600 can be configured to perform various embodiments as described herein. The user input device 600 can be used with the user input device 600 as described above. Figure 4 and Figure 5B-5C The user input device 600 may be similar or identical to the user input device 130 described above. Thus, the user input device 600 may be used in conjunction with or included in the system 100 or the systems 500 and 550. As described above, the systems 100, 500 and 550 may have the ability to determine the six-axis position and orientation of the user input device 600. Note that this includes the X, Y, Z position of the tip 610 of the user input device 600 and the α, β, γ angular orientations of the body 602 of the user input device 600. However, it should be further noted that the user input device 600 is exemplary and other user input devices of appropriate configuration may be used as desired.

[0137] As shown, the user input device 600 may include buttons 604, 606, and 612. One of the buttons, such as button 606, may be pressed and held to trigger the selection of an object in a 3D scene presented by any one of the systems 100, 500, and 550. In addition, the system 100 may be configured to display a virtual "laser-like" projection from the tip 610 to the selected object. In the case of selecting an object, adjustments to the position and / or orientation of the user input device 600 may change the position and / or orientation of the object. Therefore, movement of the user input device 600 may result in a corresponding translation and / or rotation of the object.

[0138] Replace a 2D image with a substantially equivalent 3D image

[0139] Existing web pages (e.g., hosted on a website) typically include (or contain, e.g., have a web page structure) two-dimensional (2D) images. However, as technology develops and augmented reality (AR) and / or virtual reality (VR) become more popular and available, there is a lot of evidence that if some or all of the images are presented as three-dimensional (3D) images, significant advantages can be achieved. For example, 3D images typically attract more attention than 2D images. Therefore, users (or visitors) of the web page may spend extra time on a web page based on displaying 3D images rather than 2D images. In addition, 3D images typically allow longer display content retention than 2D images. Therefore, replacing 2D images with substantially equivalent 3D images can allow the web page to attract more visitors, who may spend more time on the web page with greater content retention, compared to not replacing the 2D image.

[0140] In order to complete the conversion of 2D images to 3D images, 2D images need to be identified in the 2D web page, and the display device needs to be able to display 3D images. To do this, two things are needed. In some embodiments, the conversion from 2D images to 3D images may include determining the content of the 2D image, obtaining a 3D image with substantially equivalent content, and replacing the 2D image with a substantially equivalent 3D image (e.g., at least partially based on the determination of the 2D image content). This article discloses an embodiment of a system for implementing a method for replacing a 2D image in a web page with a substantially equivalent 3D image. The 3D image can be derived from an alternative 3D image. The alternative 3D image can be obtained from the website being visited (e.g., a hosting web page) and / or from other places on the Internet. In addition, this article discloses an embodiment of a system for implementing a method for determining and / or selecting (e.g., at least partially based on one or more criteria, such as the content of the 2D image) an optimal (e.g., substantially equivalent) 3D replacement image.

[0141] Figure 7 A block diagram illustrating an example of a process for identifying an equivalent 3D image and replacing a 2D image with it, according to some embodiments. Figure 7 The processing shown in the figure can be used with any system or device shown in the above figures and other devices. In various embodiments, some of the processing elements shown can be executed concurrently in an order different from the order shown, or can be omitted. Additional processing elements can also be executed as needed. As shown in the figure, the processing can be operated as follows.

[0142] At 702, the content of a 2D image within a web page can be identified. For example, the identification can include identification of metadata associated with the 2D image. Alternatively or additionally, if the metadata is not (readily) available, a machine learning algorithm can be used to identify the content of the 2D image, for example via analysis (or scanning) of the web page structure.

[0143] At 704, once the content of the 2D image has been identified, a 3D image with substantially equivalent content can be identified (or located). For example, if the webpage being accessed has a suitable alternative 3D image (e.g., a substantially equivalent 3D image) available, the process can select the alternative 3D image. Alternatively, if the webpage does not provide a 3D image as an alternative, a suitable 3D image (e.g., a substantially equivalent 3D image) can be obtained (e.g., searched and located / found) from elsewhere in the Internet. For example, when searching the Internet, a suitable 3D image can be identified (e.g., found or located in the Internet) based at least in part on metadata associated with the 3D image. Alternatively, if a suitable 3D image with identifying metadata is not found in the Internet, machine learning can be used to analyze the content of the 3D image found in the Internet. As another example, the content of the 3D image in the Internet can be indexed, and a suitable 3D image replacement can be identified based at least in part on a search of the indexed content of the 3D image. As another example, a 2D video or one (or more) frames of a 3D video can include content suitable for obtaining a suitable 3D image. Note that combining several video frames can improve the quality of matching a suitable 3D image to a 2D image. Other examples may include applying other and / or additional criteria to the process of selecting a suitable 3D image to replace a 2D image. Such criteria may include details of the shape of the 3D image and / or the lighting of the 3D image.

[0144] In addition, such criteria may include information from the user of the web page. For example, the user may be asked (or surveyed) to obtain information to help select a suitable 3D image to replace the 2D image. For example, if there is a 2D image of a dog on the web page, and an attempt is made to replace the 2D image of the dog with a suitable 3D image of the dog, the application may survey the user's interest in a particular breed, color, size, and / or age of the dog. Additionally (or alternatively), the application may derive (or further enhance) such additional information based on the user's browsing history and / or other online history.

[0145] At 706, a 3D image may be integrated into a web page, for example, a 3D image may be displayed in the web page instead of a 2D image. In addition, the user may interact with the integrated 3D image. For example, the 3D image may be redirected, translated (e.g., moved in any direction along any combination of the X, Y, Z axes), rotated (e.g., around any combination of the X, Y, Z axes in any direction), and the like. In other words, the 3D image may be capable of redirecting, translating, and / or other types of manipulation. (One or more) manipulations may be responsive to user input via a user input device (e.g., such as the user input devices 130 and 600 described above) and / or (e.g., in real time) information derived from the user's head and / or eye tracking as described above. In some embodiments, the information derived from the user's head and / or eye tracking may include detecting movement (e.g., change) of the user's viewpoint (or observation point).

[0146] Fig. 8A -D illustrates an example of one possible manipulation of a 3D image according to some embodiments. Fig. 8A As shown, web page 800 may include 2D image 840. 2D image 840 may be provided by, for example, Figure 7 The described process is used to identify and appropriate or equivalent 3D image 845 (such as Figure 8B As shown) can be identified (e.g., via metadata association and / or via the Internet). Figure 8C As shown, 3D image 845 can replace 2D image 840, but 3D image 845 can be manually and / or automatically reoriented so that the aspect ratio of 3D image 845 is equivalent to the aspect ratio of 2D image 840. In some embodiments, 3D image 845 can be flattened when inserted into web page 800. In other words, the value of the z-scale component of 3D image 845 can be reduced so that 3D image 845 looks like a 2D image. In some embodiments, as shown in FIG. Fig.8D As shown, once the user intersects the 3D image 845 using a user input device (e.g., stylus 130 / 600), the z-scale component of the 3D image 845 can be animated back to its original value, so that it appears 3D. Note that the z-axis can be an axis associated with the depth of the 3D image (e.g., entering or leaving the plane defined by the display), while the x-axis and y-axis can be associated with in-plane dimensions (e.g., in the plane of the display) and / or movement. Such animation can create a seamless animation effect of an object from 2D to 3D when hovering (and back to 2D when the hover is removed).

[0147] Fig. 9 A block diagram illustrating an example of a process for flattening a 3D image when integrated into a web page to replace a 2D image in accordance with some embodiments. Fig. 9The processing shown in the figure can be used with any system or device shown in the above figures and other devices. In various embodiments, some of the processing elements shown can be executed concurrently in an order different from the order shown, or can be omitted. Additional processing elements can also be executed as needed. As shown in the figure, the processing can be operated as follows.

[0148] At 902, the 2D image displayed on the web page may be replaced with a 3D image. In other words, as described above with reference to Figure 7 As described above, 2D images can be recognized, appropriate 3D images can be located, and appropriate 3D images can be integrated into web pages.

[0149] At 904, after being inserted into the web page, the 3D image may be flattened, for example, by reducing the value of the z-scale component of the 3D image. Thus, the 3D image may look like a 2D image when inserted into the web page.

[0150] At 906, upon detecting the intersection of the user input device (or the projection beam of the user input device) with the 3D image and / or the manipulation of the 3D image, the 3D image is animated by increasing the value of the z-scale component of the 3D image, thereby creating the effect of the 3D imagery that seamlessly animates from 2D to 3D when hovering. In addition, since the user input device (or the projection beam of the user input device) no longer intersects the 3D image, the 3D image is restored to a flat state by decreasing the value of the z-scale component of the 3D image. In other words, when the user input device is no longer hovering over the 3D image, the 3D image can return to being displayed as a 2D image.

[0151] Fig. 10A -C illustrates an example of another possible manipulation of a 3D image according to some embodiments. Fig. 10A As shown, web page 1000 may include 2D image 1040. 2D image 1040 may be provided by, for example, Figure 7 The described process is used to identify and appropriate or equivalent 3D image 1045 (such as Fig. 10B As shown) can be identified (e.g., via metadata association and / or via the Internet). Fig. 10C As shown, the equivalent 3D image can be presented at real-world scale / size. In other words, if the actual real-world size of the object represented by the 3D image (e.g., a soda can or soda bottle) fits within the boundaries of the application / browser viewport, then the 3D image can be provided to the user at real-world scale. In some embodiments, for example, in response to user input manipulating the 3D image, before being displayed at real-world scale, for example, as described above with reference to Figures 8A-8D and Fig. 9As described, the 3D image may first be scaled to the 2D image size and flattened. In addition, in some embodiments, when it is determined that the user is no longer interacting with the 3D image, the 3D image may be rescaled to the 2D image size. In applications such as e-commerce, representations of real-world size objects may be beneficial. In some embodiments, a customer who is shopping may view a 3D representation of an item (e.g., a 3D object) in the correct physical proportion (approximately 1:1 correspondence between the real-world size and the virtual-world size). For example, a user may be buying a ring, so a 3D virtual representation of the ring may be presented to the user, and the user may raise a finger near the display and / or the 3D representation of the ring to determine whether the ring fits and how well the ring fits. Similarly, a user may be buying a bracelet, so a 3D representation of the bracelet may be presented to the user, and the user may raise a wrist near the display and / or the 3D representation of the bracelet to determine whether the bracelet fits and how well the bracelet fits. As another example, a user may be shopping for glasses (e.g., glasses, sunglass frames, and / or lenses) and thus a 3D representation of the glasses may be presented to the user and the user may be able to visualize what the glasses would look like, for example, by positioning the glasses on their face (e.g., through user input and manipulation of the 3D representation of the glasses) and presenting a virtual mirror in the virtual 3D scene that provides a reflection of the user wearing the virtual glasses.

[0152] As described above, the selection of a suitable (or equivalent) 3D image may be based on criteria associated with the image. For example, in some embodiments, the criteria may include a ranking system. Thus, the ordered list of replacement 3D images may be ranked based at least in part on: (1) the number of times the 3D image has been previously selected as a replacement 3D image; (2) the number of votes or endorsements the 3D image has received from users who have previously searched for 3D images of this type; and / or (3) the fees paid by the 3D content creator to have the 3D image ranked higher than non-paid (less paid) 3D content creators. Additionally, the ranking may also be based at least in part on: (1) the level of detail in the 3D image; (2) whether the 3D image contains information related to the interior of a 3D object displayed within the 3D image; and / or (3) the size of the 3D image.

[0153] Fig.11 A block diagram illustrating an example of a process for selecting an appropriate 3D image to replace a 2D image displayed on a web page in accordance with some embodiments. Fig.11 The processing shown in the figure can be used with any system or device shown in the above figures and other devices. In various embodiments, some of the processing elements shown can be executed concurrently in an order different from the order shown, or can be omitted. Additional processing elements can also be executed as needed. As shown in the figure, the processing can be operated as follows.

[0154] At 1102, the content of a 2D image within a web page can be identified. For example, the identification can include identification of metadata associated with the 2D image. Alternatively or additionally, if the metadata is not (readily) available, a machine learning algorithm can be used to identify the content of the 2D image, for example via analysis (or scanning) of the web page structure.

[0155] At 1104, once the content of the 2D image has been identified, a plurality of 3D images having content similar to the 2D image may be identified (or located). In other words, each of the plurality of 3D images may have imagery, metadata, image tags, etc. that are related to or also associated with the 2D image. For example, if the accessed web page has a suitable alternative 3D image available (e.g., a substantially equivalent 3D image), the process may identify the alternative 3D image. Alternatively, if the web page does not provide the 3D image as an alternative, the 3D image may be obtained (e.g., searched and located / found) from elsewhere in the Internet. For example, when searching the Internet, the 3D image may be identified (e.g., found or located in the Internet) based at least in part on metadata associated with the 3D image. Alternatively, if no 3D image with identifying metadata is found in the Internet, machine learning may be used to analyze the content of the 3D image found in the Internet. As another example, the content of the 3D image in the Internet may be indexed, and the 3D image replacement may be identified based at least in part on a search of the indexed content of the 3D image. As another example, one (or more) frames of a 2D video or a 3D video may include content suitable for acquiring a 3D image. Note that combining several video frames can improve the matching quality of a 3D image with a 2D image. Other examples may include applying other and / or additional criteria to the process of selecting a 3D image as a possible replacement for a 2D image. Such criteria may include the shape of the 3D image and / or the details of the lighting of the 3D image. In addition, in some embodiments, if a fully matched (e.g., suitable) 3D image is not found, a 3D reconstruction algorithm may be applied to a 2D image and / or a 2D video (continuous footage) to generate a reconstructed 3D model, which may then be used as a suitable 3D image for replacement in a web page.

[0156] At 1106, the plurality of 3D images may be ranked, for example, based on one or more sets of ranking criteria. For example, a first set of ranking criteria may include: (1) the number of times the 3D image has been previously selected as a replacement 3D image; (2) the number of votes or endorsements the 3D image has received from users who have previously searched for 3D images of this type; and / or (3) fees paid by the 3D content creator to rank the 3D image higher than non-paid (less paid) 3D content creators. Additionally, a second set of ranking criteria may include: (1) the level of detail in the 3D image; (2) whether the 3D image contains information related to the interior of a 3D object displayed within the 3D image; and / or (3) the size of the 3D image. Note that the ranking of the plurality of 3D images may combine multiple sets of criteria and / or may consider multiple sets of criteria separately. In some embodiments, one set of criteria may be weighted compared to another set of criteria. Such weighting may result in a tie if the ranking from one set of criteria produces a different result than the ranking from another set of criteria. In some embodiments, such weighting may be used to combine rankings from various sets of criteria.

[0157] At 1108, the highest ranked 3D image may be selected as an equivalent 3D image. Once an equivalent (or suitable) 3D image is selected, the equivalent 3D image may be integrated into a web page, for example, the equivalent 3D image may be displayed in a web page instead of a 2D image. In addition, the user may interact with the integrated equivalent 3D image. For example, the equivalent 3D image may be redirected, translated (e.g., moved in any direction along any combination of the X, Y, Z axes), rotated (e.g., around any combination of the X, Y, Z axes in any direction), and the like. In other words, the equivalent 3D image may be capable of redirection, translation, and / or other types of manipulation. (One or more) manipulations may be responsive to user input via a user input device (e.g., such as the user input devices 130 and 600 described above) and / or (e.g., in real time) information derived from the user's head and / or eye tracking as described above.

[0158] In addition to selecting a suitable (or equivalent) 3D image, the quality or suitability of a particular 3D image can also be improved. For example, once the highest ranked suitable 3D image is identified or selected, an image can be added to the highest ranked suitable 3D image. The image can be derived (or can be derived) from the 2D image replaced by the highest ranked suitable 3D image. As another example, once the highest ranked suitable 3D image is identified or selected, the incorrect 3D image in the highest ranked suitable 3D image can be replaced with the correct 3D image derived (or can be derived) from the 2D image. In other words, the 3D image that does not correspond to the 2D image in the 2D image can be removed and / or modified so that the corrected 3D image does correspond to the 2D image. As another example, once the highest ranked suitable 3D image is identified or selected, the 2D image content can be mapped to the 3D image content to replace the existing image in the highest ranked suitable 3D image. In addition, a suitable 3D image can be created (or derived) by combining aspects of two or more less suitable 3D images. In other words, if there is no suitable 3D image, a suitable 3D image may be constructed (or assembled) based on two or more found 3D images.

[0159] Furthermore, in some embodiments, an initial pose of a suitable 3D image model may be rendered such that the orientation, scale, and position of the model (e.g., in the directions “x” and “y” in the plane) may match the orientation, scale, and position (in “x” and “y”) of the 2D image to be replaced, thereby creating a seamless user experience in the replacement process from a 2D image to a suitable 3D image.

[0160] Fig.12 A block diagram illustrating an example of a process for improving the suitability of a 3D image to replace a 2D image displayed on a web page in accordance with some embodiments. Fig.12 The processing shown in the figure can be used with any system or device shown in the above figures and other devices. In various embodiments, some of the processing elements shown can be executed concurrently in an order different from the order shown, or can be omitted. Additional processing elements can also be executed as needed. As shown in the figure, the processing can be operated as follows.

[0161] At 1202, a selected equivalent 3D image (3D image) may be analyzed for accuracy relative to the 2D image that the 3D image is replacing. For example, imagery (e.g., background or background objects, foreground or foreground objects, lighting, shading, shadows, etc.) of the 3D image may be compared to imagery of the 2D image.

[0162] At 1204, an image of the 3D image that does not correspond to the image of the 2D image may be identified, and at 1206, the identified image may be modified and / or replaced. For example, the image may be added to the 3D image. In some embodiments, the added image may be derived (or exportable) from the 2D image. As another example, the 2D image content (image) may be mapped onto the 3D image content to replace the existing image in the 3D image.

[0163] At 1208, a modified 3D image may be displayed. In other words, the modified 3D image may be integrated into a web page, e.g., the modified 3D image may be displayed in a web page instead of a 2D image. In addition, the user may interact with the modified 3D image. For example, the modified 3D image may be redirected, translated (e.g., moved in any direction along any combination of the X, Y, Z axes), rotated (e.g., around any combination of the X, Y, Z axes in any direction), and the like. In other words, the modified 3D image may be capable of redirection, translation, and / or other types of manipulation. (One or more) manipulations may be responsive to user input via a user input device (e.g., the user input devices 130 and 600 described above) and / or (e.g., in real time) information derived from user head and / or eye tracking as described above.

[0164] Embodiments of the present disclosure may be implemented in any of a variety of forms. For example, some embodiments may be implemented as computer-implemented methods, computer-readable storage media, or computer systems. Other embodiments may be implemented using one or more custom designed hardware devices (such as ASICs). Other embodiments may be implemented using one or more programmable hardware elements (such as FPGAs).

[0165] In some embodiments, a non-transitory computer-readable storage medium may be configured such that it stores program instructions and / or data, wherein if the program instructions are executed by a computer system, the program instructions cause the computer system to perform a method, for example, any method embodiment described herein, or any combination of the method embodiments described herein, or any subset of any method embodiment of the method embodiments described herein, or any combination of such subsets.

[0166] In some embodiments, a wireless device (or wireless station) may be configured to include a processor (or a group of processors) and a storage medium, wherein the storage medium stores program instructions, wherein the processor is configured to read the program instructions from the storage medium and execute the program instructions, wherein the program instructions are executable to cause the wireless device to implement any one of the various method embodiments described herein (or any combination of the method embodiments described herein, or any subset of the method embodiments described herein, or any combination of such subsets). The device may be implemented in any of a variety of forms.

[0167] Although the embodiments have been described above in considerable detail, numerous changes and modifications will become apparent to those skilled in the art once the above disclosure is fully appreciated. It is intended that the following claims be interpreted to encompass all such changes and modifications.

Claims

1. A non-transitory computer-readable storage medium storing programming instructions executable by processing circuitry of a wireless device to: identifying content of a two-dimensional (2D) image displayed within a web page on at least one display of the wireless device, wherein the 2D image is identified based on one of a structural analysis of the web page or an identification of metadata associated with the 2D image, and wherein the web page is displayed to a user; Based at least in part on the content of the 2D image, identifying one or more three-dimensional (3D) images as possible replacements for the 2D image; ranking the one or more 3D images based on one or more sets of ranking criteria, wherein a first set of ranking criteria includes a number of times the 3D image has been previously selected as a replacement 3D image, a number of votes the 3D image has received from users who have previously searched for 3D images of this type, a number of endorsements the 3D image has received from users who have previously searched for 3D images of this type, and a fee paid by the 3D content creator to have the 3D image ranked higher than less paid 3D content creators, and wherein a second set of ranking criteria includes a level of detail in the 3D image compared to other 3D images, whether the 3D image contains information related to the interior of a 3D object displayed within the 3D image, and a size of the 3D image; selecting the 3D image having the highest ranking value; and Integrate 3D images into web pages, thereby replacing 2D images with 3D images.

2. The non-transitory computer-readable storage medium of claim 1, wherein: The programming instructions are also executable to: Receiving at least one user input for manipulating a 3D image within a webpage, wherein the at least one user input comprises at least one of: movement of a user's point of view relative to the at least one display, wherein the movement of the point of view is determined based at least in part on tracking information of the user's head or eyes; or Detection of a light beam projected from an end of the user input device that intersects the 3D image, wherein the detection of the light beam that intersects the 3D image is based at least in part on a tracked position and orientation of the user input device relative to the at least one display.

3. The non-transitory computer-readable storage medium of claim 1, wherein: To identify the one or more images, the programming instructions are further executable to: One or more 3D images hosted by a website hosting a web page are selected.

4. The non-transitory computer-readable storage medium of claim 1, wherein: To identify the one or more images, the programming instructions are further executable to: Select one or more images identified through an Internet search.

5. The non-transitory computer-readable storage medium of claim 4, wherein: To select the one or more images identified through the Internet search, the programming instructions are further executable to: One or more 3D images are identified based at least in part on metadata associated with the 3D images.

6. The non-transitory computer-readable storage medium of claim 4, wherein: To select the one or more images identified through the Internet search, the programming instructions are further executable to: Search the content index of 3D images.

7. The non-transitory computer-readable storage medium of claim 1, wherein: The one or more sets of ranking criteria are combined into a single set of ranking criteria.

8. The non-transitory computer-readable storage medium of claim 1, wherein: The values ​​of the first ranking based on the first set of ranking criteria are weighted compared to the values ​​of the second ranking based on the second set of ranking criteria.

9. The non-transitory computer-readable storage medium of claim 1, wherein: For each 3D image of the one or more 3D images, the programming instructions are further executable to: A ranking value is determined, wherein the ranking value is based at least in part on the ranking value of each of the one or more sets of ranking criteria.

10. The non-transitory computer-readable storage medium of claim 9, wherein: The ranking value is a weighted combination of the ranking values ​​of each of the one or more sets of ranking criteria.

11. The non-transitory computer-readable storage medium of claim 1, wherein: The programming instructions are also executable to: Analyze the accuracy of 3D images relative to 2D images; identifying an image of the 3D image that does not correspond to an image of the 2D image; and The identified image is modified so that the modified image of the 3D image corresponds to the image of the 2D image.

12. A three-dimensional (3D) stereoscopic display system, comprising: at least one processor; one or more displays coupled to at least one processor; as well as a memory coupled to the at least one processor, wherein the at least one processor is configured to: identifying content of a two-dimensional (2D) image displayed within a web page displayed on at least one of the one or more displays, wherein the web page is displayed to a user; identifying one or more three-dimensional (3D) images as possible replacements for the 2D image based on the content of the 2D image through a web page structure search or Internet search; selecting the 3D image having the highest ranking value, wherein the highest ranking is determined from the ranking of the one or more 3D images based on one or more sets of ranking criteria, wherein a first set of ranking criteria includes a number of times the 3D image has been previously selected as a replacement 3D image, a number of votes the 3D image has received from users who have previously searched for 3D images of this type, a number of endorsements the 3D image has received from users who have previously searched for 3D images of this type, and a fee paid by the 3D content creator to have the 3D image ranked higher than less expensive 3D content creators, wherein a second set of ranking criteria includes a level of detail in the 3D image compared to other 3D images, whether the 3D image contains information related to the interior of a 3D object displayed within the 3D image, and a size of the 3D image; identifying an image of the 3D image that does not correspond to an image of the 2D image; modifying the identified image so that the modified image of the 3D image corresponds to the image of the 2D image, thereby generating a modified 3D image; as well as The modified 3D image is integrated into the web page, thereby replacing the 2D image with the modified 3D image.

13. The 3D stereoscopic display system according to claim 12, wherein the at least one processor is further configured to: Receiving at least one user input for manipulating a 3D image within a webpage, wherein the at least one user input comprises at least one of: movement of a user's point of view relative to the at least one display, wherein the movement of the point of view is determined based at least in part on tracking information of the user's head or eyes; or Detection of a light beam projected from an end of the user input device that intersects the 3D image, wherein the detection of the light beam that intersects the 3D image is based at least in part on a tracked position and orientation of the user input device relative to the at least one display.

14. The 3D stereoscopic display system according to claim 12, wherein: The ranking value is a weighted combination of the ranking values ​​of the first set of ranking criteria and the second set of ranking criteria.

15. The 3D stereoscopic display system according to claim 12, wherein: The one or more sets of ranking criteria are combined into a single set of ranking criteria.

16. The 3D stereoscopic display system of claim 12, wherein the image comprises at least two of the following items: The shape of the 3D image; Background objects; Foreground Object illumination; Coloring; or shadow.

17. The 3D stereoscopic display system according to claim 12, wherein: The modified imagery includes an imagery derived from the 2D image.

18. The 3D stereoscopic display system according to claim 12, wherein: The modified imagery includes an imagery mapped from a 2D image to a 3D image.

19. A wireless device comprising: at least one antenna; at least one radio in communication with the at least one antenna and configured to perform wireless communications using at least one radio access technology (RAT); one or more processors coupled to the at least one radio; as well as at least one display in communication with the one or more processors; wherein the one or more processors are configured to cause the wireless device to: identifying content of a two-dimensional (2D) image displayed within a web page on a display of the wireless device, wherein the web page is displayed to a user; identifying a three-dimensional (3D) image having substantially equivalent content compared to the content of the 2D image; selecting the 3D image having the highest ranking value, wherein the highest ranking is determined based on the ranking of one or more 3D images identified as possible replacements for the 2D image based on one or more sets of ranking criteria, wherein a first set of ranking criteria includes a number of times the 3D image has been previously selected as a replacement 3D image, a number of votes the 3D image has received from users who have previously searched for 3D images of this type, a number of endorsements the 3D image has received from users who have previously searched for 3D images of this type, and a fee paid by the 3D content creator to have the 3D image ranked higher than less expensive 3D content creators, wherein a second set of ranking criteria includes a level of detail in the 3D image compared to other 3D images, whether the 3D image contains information related to the interior of a 3D object displayed within the 3D image, and a size of the 3D image; identifying an image of the 3D image that does not correspond to an image of the 2D image; modifying the identified image so that the modified image of the 3D image corresponds to the image of the 2D image, thereby generating a modified 3D image; integrating the modified 3D image into the web page, thereby replacing the 2D image with the modified 3D image; and Receiving at least one user input for manipulating a 3D image within a webpage, wherein the at least one user input comprises at least one of: movement of a user's point of view relative to the at least one display, wherein the movement of the point of view is determined based at least in part on tracking information of the user's head or eyes; or Detection of a light beam projected from an end of the user input device that intersects the 3D image, wherein the detection of the light beam that intersects the 3D image is based at least in part on a tracked position and orientation of the user input device relative to the at least one display.

20. The wireless device of claim 19, wherein: The ranking value is a weighted combination of the ranking values ​​of the first set of ranking criteria and the second set of ranking criteria.

21. The wireless device of claim 19, wherein: The one or more sets of ranking criteria are combined into a single set of ranking criteria.

22. The wireless device of claim 19, wherein: The image includes at least two of the following: The shape of the 3D image; Background objects; Foreground Object illumination; Coloring; or shadow.

23. The wireless device of claim 19, wherein: The modified imagery includes an imagery derived from the 2D image.

24. The wireless device of claim 19, wherein: The modified imagery includes an imagery mapped from a 2D image to a 3D image.

Images