Augmented Reality Rendering
By generating encoded images on desktop computing devices and decoding presenting augmented reality data on mobile devices, a seamless augmented reality experience and a consistent user interface is achieved.
Patent Information
- Application Number
- CN202080094370.X
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Priority Date
- 2020-09-09
- Filing Date
- 2020-09-10
- Publication Date
- 2025-08-01
- Estimated Expiration
- 2040-09-10
AI Technical Summary
The prior art cannot achieve seamless conversion to an augmented reality experience on desktop computing devices, especially online conversations on desktop computing devices that cannot be converted directly on augmented reality-enabled mobile devices.
By selecting augmented reality controls on a desktop computing device, generating encoded images, mobile devices capture and decode images, receive unified resource indicators, request augmented reality data, and present an augmented reality experience on mobile devices, including the three-dimensional model of the object and synchronization of user sessions.
A seamless transition from desktop computing devices to augmented reality-enabled mobile devices provides a consistent augmented reality experience and reduces server requests and improves consistency in load times and user experience.
Smart Images

Figure CN115004130B_ABST
Abstract
Description
Background Art
[0001] The system can use a three-dimensional (“3D”) model to represent an object. For example, an augmented reality (“AR”) system, a virtual reality (“VR”) system, or a web browser can use a 3D model to represent an object in a corresponding environment. The model can be outside the field of view and affect the rendering of an object within the field of view (e.g., a ball bouncing on a chair). Summary of the Invention
[0002] This specification describes techniques, methods, systems, and other means for presenting an augmented reality experience. The augmented reality experience can involve displaying virtual objects within a view of a real-world scene and can be performed using a mobile computing device equipped with a camera device and capable of providing such an experience. However, some online sessions that present images of objects on web resources are conducted on computing devices that cannot enable augmented reality, such as desktop computers, kiosk computers, and other such devices. For example, a web page presented by a web browser running on a desktop computing device can present an image of an object that a user wants to participate in an augmented reality experience with. In this example, the user can transfer the object, the user, session information, or a combination of these from the computing device that cannot enable augmented reality to another computing device that enables augmented reality, such as a smartphone. To transfer the information, for example, the user can select a control on the web page presented by the web browser running on the desktop computing device, and in response, the web page can generate an image encoding the object, the user, session information, or a combination of two or more of these. Then, the smartphone can use the camera device to capture the encoded image based on user input, for example, and decode the encoded information, and use the information to initiate an augmented reality experience including the object and associate the online session on the smartphone with the online session on the desktop computing device. As another example, a virtual reality experience including the object can be initiated, and the virtual reality experience facilitated by the device that enables virtual reality can be associated with the online session on the desktop computing device.
[0003] Generally, an innovative aspect of the subject matter described in this specification can be embodied in a method that includes the following actions: receiving, by a mobile device web browser, data of a web page, the data of the web page being associated with a uniform resource identifier and including augmented reality data for a plurality of augmented reality presentations; selecting, by the mobile device web browser, an augmented reality presentation from the plurality of augmented reality presentations of the web page by: (i) parsing the uniform resource identifier, (ii) detecting one or more augmented reality presentation parameters in the parsed uniform resource identifier, and (iii) selecting, from the augmented reality data for the plurality of augmented reality presentations, specific augmented reality data for the augmented reality presentation identified by the one or more augmented reality presentation parameters; automatically presenting, by the mobile device web browser, the augmented reality presentation of the web page using the specific augmented reality data for the augmented reality presentation in the plurality of augmented reality presentations and data from a camera device included in the mobile device on which the mobile device web browser is running; after automatically presenting the augmented reality presentation, receiving, by the mobile device web browser, data indicating that the augmented reality presentation should end; and after receiving the data indicating that the augmented reality presentation should end, presenting, by the mobile device web browser, the web page using the data of the web page.
[0004] Other implementations of this aspect include corresponding computer systems, apparatuses, computer program products, and computer programs recorded on one or more computer storage devices, each configured to perform the actions of the method. A system of one or more computers can be configured to perform particular operations or actions by means of software, firmware, hardware, or a combination thereof installed on the system, the software, firmware, hardware, or a combination thereof causing the system to perform the actions in operation. One or more computer programs can be configured to perform particular operations or actions by means of instructions that, when executed by a data processing apparatus, cause the apparatus to perform the actions.
[0005] The foregoing and other embodiments may each optionally include, individually or in combination, one or more of the following features. Receiving data for a web page may include receiving data independent of the underlying operating system of the mobile device. Augmented reality data may be specific to the mobile device, the user's account of the mobile device, or both. The augmented reality data may include a three-dimensional model of an object. One or more augmented reality rendering parameters may include a rendering angle parameter for specifying an initial rendering angle of the three-dimensional model of the object. One or more augmented reality rendering parameters may include a color parameter for specifying an initial rendering color of the three-dimensional model of the object. One or more augmented reality rendering parameters may include a size parameter for specifying an initial rendering size of the three-dimensional model of the object. Image data encoding a uniform resource identifier may be received from the imaging device of the mobile device. The mobile device may determine the uniform resource identifier based on the image data and provide the uniform resource identifier to the mobile device web browser. The image data may be determined based on an image visually presented by a computing device different from the mobile device and captured by the imaging device of the mobile device. The uniform resource identifier encoded in the image data visually presented by the computing device may include a session identifier, and the mobile device web browser may associate a first user session of the computing device with a second user session on the mobile device. An augmented reality rendering may be automatically presented in response to determining that the mobile device is enabled to generate an augmented reality rendering. After receiving data indicating that the augmented reality rendering should end, rendering the web page may occur without the mobile device requesting additional content from the server system. For each of two or more web page components, a component may be generated by the mobile device web browser. For a first web page component of the two or more web page components, an identifier of the web page component in the uniform resource identifier may be detected. The detected identifier of the web page component in the uniform resource identifier may be used to select an augmented reality rendering of the first web page component. For a second web page component, analysis of the uniform resource identifier may be determined to be skipped in response to the determination of the first web page component.
[0006] The subject matter described in this specification can be implemented in various embodiments and can produce one or more of the following advantages. In some implementations, the features described in this document improve the system, for example, by providing complete web page content (e.g., along with supplementary content) to a client device, which can then dynamically adjust the content presented based on the URL while still having the option to present a "basic" web page. For example, the URL can indicate that the client should load the basic web page and initiate an AR experience, and after loading and initiating, the client device can present the basic web page without requesting additional content from the server. In some implementations, for example, client devices that load web pages in the background while presenting an AR experience based on a single URL improve the loading time. In some implementations, a consistent web codebase can be maintained on a server, for example, by using the same web page content to render web pages on different computing devices and / or associating augmented reality presentations with controls (e.g., buttons) for initiating object-specific augmented reality experiences on different web pages, while providing a consistent and flexible user experience between different devices, efficient application of user choices, or both. For example, this can reduce the size of the web codebase that needs to be maintained. In some implementations, for example, by using encoded images to connect user sessions conducted on different computing devices, the system can apply analytical techniques for attributing website conversions, determining user interactions across multiple different devices, or both. In some implementations, a one-touch conversion from a desktop web browsing experience to an AR experience can be provided. For example, the one-touch conversion can include not only opening a link from an encoded image captured by a camera device but also directly opening an AR experience.
[0007] Details of one or more implementations of the subject matter described in this specification are set forth in the accompanying drawings and the following description. Other features, aspects, and advantages of the subject matter will become apparent from the description, the drawings, and the claims. BRIEF DESCRIPTION OF THE DRAWINGS
[0008] Figure 1 is an example environment for presenting an augmented reality experience.
[0009] Figure 2 is a flowchart of an example process for presenting an augmented reality experience.
[0010] Figures 3A to 3B is an example interface for selecting an augmented reality experience.
[0011] Figures 4A to 4B is an example interface for presenting an augmented reality experience.
[0012] Figure 5 is a block diagram of a computing system that can be used in conjunction with the computer-implemented methods described in this document.
[0013] Like reference numerals and names in the various figures indicate like elements. DETAILED DESCRIPTION
[0014] An augmented reality experience can involve presenting virtual objects within a view of a real-world scene and can be performed using a mobile computing device equipped with a camera device and capable of providing such an experience. However, some online sessions that present images of objects on web resources are conducted on computing devices that are not capable of enabling augmented reality, such as desktop computers, kiosk computers, and other such devices. The techniques described in this document can provide techniques for transferring objects, users, session information, or combinations thereof for an online session from a device not capable of enabling an augmented reality experience to an augmented reality-enabled device in an almost seamless manner.
[0015] Figure 1 is an example environment 100 for presenting an augmented reality experience. In other examples, environment 100 can be configured to present a virtual reality experience. In the depicted example, environment 100 includes one or more computer servers 102 that communicate with one or more online computing devices 104 and one or more mobile computing devices 106. Communication between computing servers 102, online computing devices 104, and mobile computing devices 106 can occur, for example, via one or more communication networks (not shown) including a LAN (local area network), a WAN (wide area network), and / or the Internet.
[0016] In some examples, computing server 102 can represent various forms of servers, including but not limited to web servers, application servers, cloud computing servers, or other suitable computing servers. Computing server 102 can, for example, access data from various sources (e.g., databases, file systems, and / or cached data sources), execute software that processes the accessed data, and provide information based on the accessed / processed data to online computing device 104 and / or mobile computing device 106. For example, computing server 102 can access data from website data source 110, which includes data for providing one or more websites, such as web page data, user history data; data for providing an augmented reality experience, and other suitable data.
[0017] In some examples, the online computing device 104 may represent various forms of fixed or mobile processing devices, including but not limited to desktop computers, kiosk computers, laptop computers, tablet computers, personal digital assistants (PDAs), smart phones, or other processing devices. In some examples, the mobile computing device 106 may represent various forms of mobile processing devices, including but not limited to tablet computers, personal digital assistants (PDAs), smart phones, head-mounted computers, or other processing devices. The online computing device 104 and the mobile computing device 106 may, for example, run a web browser that may execute a script providing resource data.
[0018] Figure 1 Also shown is an example processing flow for presenting an augmented reality experience and / or a virtual reality experience as shown in phases (A) through (E). Phases (A) through (E) may occur in the order shown, or they may occur in an order different from the order shown, and / or two or more of phases (A) through (E) may be simultaneous or partially simultaneous.
[0019] Now referring to Figure 2 , an example process 200 for presenting an augmented reality experience is shown. In other examples, process 200 may present a virtual reality experience. For example, process 200 may be performed by components of environment 100 and will be described with reference to Figure 1 . However, other systems may be used to perform the same or similar processes.
[0020] At (210), a computing device 204 (e.g., similar to the Figure 1 online computing device 104 shown) may request a web page from a web server 202 (e.g., similar to the Figure 1 computing server 102 shown). For example, a user of the computing device 204 may provide an input identifying the web page (e.g., enter a uniform resource locator (“URL”) using a physical or virtual keyboard, specify an address or keyword of an online resource using a microphone, select a resource link using a pointing device or touch screen, or use another input device), and the computing device 204 may provide the input to a web browser that requests the web page from the web server 202. In this example, the requested web page may be a page from an e-commerce website, however in other examples other kinds of web resources may be requested.
[0021] At (212), the web server 202 may receive the request and, in response, may send web page data (214) to the computing device 204. Again referring to Figure 1, for example, during stage (A), computing server 102 can access website data storage device 110, retrieve web page content 112 of the requested web page, and provide web page content 112 to online computing device 104 in response to the request.
[0022] At (216), computing device 204 can present the requested web page. Referring again to Figure 1 , for example, online computing device 104 can use the received web page content 112 to render web page 120 for presentation on the device display.
[0023] Now referring to Figure 3A , an example interface 300 for selecting an augmented reality experience (e.g., similar to web page 120) is shown. In this example, interface 300 includes various controls 310 facilitating an online shopping session by a user (e.g., user A), such as one or more controls for logging in or out of the website, accessing an online shopping cart, adding items to the cart, favoriting items, etc.
[0024] Interface 300 in this example also includes image selection controls 312a and 312b. For example, by interacting with image selection controls 312a, 312b, a user can switch between various images 314 of an object (e.g., a chair) presented on interface 300, where the currently selected image of the object is presented in more detail at image presentation area 320. As another example, a user can directly select (e.g., click) one of the images 314 of the object, and the selected image of the object can be presented in more detail at image presentation area 320. In this example, the various selectable images 314 of the object include different presentation angles (e.g., front, side, back, etc.), however in other examples, the various selectable images 314 can include different colors of the object, different sizes of the object, and / or other different object attribute values. In some implementations, images of multiple different objects can be selectable on interface 300. For example, the various selectable images 314 can include images of different objects, such as images of chairs, images of tables, images of lamps, etc.
[0025] In this example, the interface 300 includes a control 330 that is used to facilitate an augmented reality experience of presenting an object (e.g., an item such as a chair) on the interface 300. In some implementations, the interface 300 may include multiple controls 330, e.g., for different objects, different colors of an object, different sizes of an object, or a combination of two or more of these. Generally, an augmented reality experience may involve the real-time display of one or more virtual objects within a view of a real-world scene. For example, a 3D model of an object may be rendered by a computing device, and a virtual image of the object may be integrated with the real-world scene (e.g., the user's environment) such that the virtual image of the object appears to the user to be part of the real-world scene.
[0026] Hardware components for implementing an augmented reality experience may include a processor, a display, and an input device (e.g., a camera device) as well as other hardware components. For example, a mobile computing device such as a tablet computer, a personal digital assistant, a smart phone, a head-mounted computer, or other mobile computing device may include: a camera device that captures real-time images of a real-world scene; a processor that renders a 3D model of an object to generate a virtual image and registers the virtual image within the real-world scene; and a display that presents a composite image including the real-world scene and the virtual image of the object. The mobile computing device may preferably be a fixed computing device to facilitate the augmented reality experience. For example, when the mobile computing device moves, the real-world scene may change, and the virtual image of the object may be continuously modified by the processor to maintain the orientation of the object relative to the scene. In addition, some fixed computing devices (e.g., desktop computers) may not be equipped with a camera device or may not otherwise be configured to facilitate an augmented reality experience. However, such fixed computing devices are often used to initiate an online shopping session. For example, by interacting with the control 330, a user can initiate a streamlined process for initiating an augmented reality experience, including transmitting the object and / or session information from a computing device (e.g., a fixed computing device) that is not configured to facilitate the experience to a computing device (e.g., a mobile computing device) that is configured to facilitate the experience.
[0027] Referring again to Figure 2 , at (218), the computing device 204 (e.g., a fixed computing device not configured to facilitate an augmented reality experience) may receive an augmented reality control selection (e.g., the user's selection of the control 330 shown in FIG. 3). In response to receiving the augmented reality control selection, e.g., the computing device 204 may present an encoded image (220).
[0028] Now referring to Figure 3B, for example, an example interface 350 for facilitating an augmented reality experience is shown (e.g., similar to web page 120 or a part of web page 120). In this example, interface 350 includes an encoded image 360 (e.g., a QR code, a smart image, or another suitable encoded image) that encodes the address of a resource (e.g., a web page), and this encoded image can provide an augmented reality experience for one or more objects (e.g., a chair) selected by the user through interface 300 ( Figure 3A as shown). As another example, for each of the one or more objects, the provided resource (e.g., a promotional email) can include a corresponding encoded image for initiating an augmented reality experience for that object. The objects included in the provided resource can be selected, for example, by a recommendation engine (not shown). As another example, a printed page can include one or more encoded images for initiating an augmented reality experience for an object.
[0029] In some implementations, the encoded image can encode one or more augmented reality presentation parameters, one or more user parameters, and / or one or more session parameters. For example, the augmented reality presentation parameters in encoded image 360 can specify an object and / or various presentation attributes of the object, such as a presentation angle, a presentation color, a presentation size, and / or other suitable presentation attributes. The presentation parameters can correspond, for example, to the attributes of an object selected by the user before starting the process for initiating an augmented reality experience. In this example, the user has selected an image of a chair at a front angle from among various selectable images 314 ( Figure 3A as shown). The user parameters and / or session parameters can include, for example, a user identifier and / or a session identifier of an online session conducted by the user on computing device 204. For example, the user can log in to an online shopping website using computing device 204, and web server 202 can generate a unique session identifier for the user's current session. As another example, the presentation parameters, user parameters, and / or session parameters can be included in other kinds of resources (e.g., in the encoded images provided in promotional emails, printed pages, etc.), such that the resources from which an augmented reality presentation has been initiated can be identified.
[0030] At (222), the encoded image can be captured by mobile device 206 (e.g., similar to the Figure 1 mobile computing device 106 shown). For example, a user of mobile device 206 (e.g., potentially the same user or a different user as that of computing device 204) can use the capture device of mobile device 206 to capture the encoded image. Referring again to Figure 1 , for example, during stage (B), mobile computing device 106 can capture web page 120 (e.g., similar to Figure 3Bthe encoded image 160 on the interface 350 (e.g., similar to the encoded image 360 also shown in Figure 3B to receive the image data 122.
[0031] At (224), the mobile device 206 can decode the captured encoded image and parse the decoded data to determine a Uniform Resource Identifier ("URI") based on the data. Referring again to Figure 1 , for example, the mobile computing device 106 can decode the image data 122 received from the imaging device 130, can determine the URI and can provide the URI to the web browser 132 of the device. For example, the web browser 132 can be a fully native web application executed by the mobile computing device 106, a web container loaded by another application executed by the device 106, or another suitable type of web browser instance.
[0032] At (226), the mobile device 206 can request a web page from the web server 202. Referring again to Figure 1 , for example, during phase (C), the web browser 132 running on the mobile computing device 106 can provide a content request 134 to one or more of the computing servers 102 based on the URI decoded from the image data 122. For example, the web browser 132 can request the resource (e.g., web page) indicated by the URI, and the URI can optionally include one or more augmented reality rendering parameters, one or more user parameters, and / or one or more session parameters.
[0033] At (228), the web server 202 can receive the request and, in response, can send web page data (230) to be received (232) by the mobile device 206. Referring again to Figure 1 , for example, during phase (D), the computing server 102 can access the web page content 136 of the requested web page in the website data source 110 and can provide the web page content 136 to the mobile computing device 106 in response to the request. The web page content 136 can, for example, include data of the web page 140 associated with the URI decoded from the image data 122. For example, the data of the web page 140 can include augmented reality data for a plurality of augmented reality renderings. In this example, the web page content 136 for rendering the web page 140 can be the same web page content 112 for rendering the web page 120 (e.g., the online shopping interface 300 depicting a chair). In some examples, the web page content 136 for rendering the web page 140 can be different from the web page content 112 for rendering the web page 120.
[0034] In some implementations, receiving data for a web page can include receiving data independent of the underlying operating system of the mobile device. For example, web page content 136 can include data for rendering a web page 140 on various different computing devices and / or for providing an augmented reality experience 142 (e.g., within the web page 140 or another application), and similarly, the same web page content 136 can be provided in response to a similar content request. For example, when processing web page content 136 to render a web page 140 and / or provide an augmented reality experience 142, the mobile computing device 106 can use a particular operating system and / or web browser to process the received data. For example, this can enable the computing server 102 to provide the same web page content 136 to different mobile computing devices 106, regardless of the underlying hardware, operating system, web browser, or combination of two or more of these of the receiving mobile computing device 106.
[0035] In some implementations, augmented reality data can be selected based on the mobile device, the mobile device user's account, or both. For example, the content request 134 can include information indicating one or more specifications of the mobile computing device 106 (e.g., model, operating system, web browser, etc.), and one or more computing servers 102 can provide web page content 136 that includes augmented reality data configured to be used on a particular device. As another example, the content request 134 can include a user account identifier (e.g., from the encoded image 160 received in the image data 122 and / or from the mobile computing device 106), and one or more computing servers 102 can provide web page content 136 that includes augmented reality data based on the user account. For example, one or more computing servers 102 can access the website data 110 and can identify the augmented reality rendering parameters (e.g., angle, color, and / or size) of an object that has been previously selected by the user and previously used by the user in an augmented reality experience, and can provide such parameters (and / or can provide a model of the object with such parameters) to the mobile computing device 106 for generating the augmented reality experience 142. For example, the user may have previously participated in an augmented reality experience with other objects (e.g., other chairs) and previously specified augmented reality rendering parameters for the other objects, such as a rendering angle (e.g., front), a rendering color (e.g., brown), and a rendering size (e.g., large). When presenting a virtual image of an object, for example, when such parameters are not explicitly specified by the encoded image 160, the previously specified rendering parameters can be applied so that the augmented reality experience for the user account can be consistent among several different objects.
[0036] In some implementations, the augmented reality data can include a three-dimensional model of an object. For example, the web content 136 including the augmented reality data can include a three-dimensional model of an object (e.g., a chair) selected by a user on the interface 300 ( Figure 3A as shown). In this example, a basic model of the object can be provided in the augmented reality data, and when presenting the virtual image of the model in the augmented reality experience 142, the basic model of the object can be rendered by the mobile computing device 106 according to various rendering parameters (e.g., angle, color, size, etc.). As another example, multiple different models of the object (e.g., Figure 3A different models for each of the various alternative images 314 shown in) can be provided in the augmented reality data, and an appropriate one of the models can be selected and rendered according to the rendering parameters. In some examples, the computing server 102 can select one of the multiple different models and include the data of the selected model in the augmented reality data. The selected model can be based on the control 330 in the interface 300 where the computing device 104 presents the encoded image 160.
[0037] In some implementations, a user session conducted on the online computing device 104 can be associated with a user session conducted on the mobile computing device 106. For example, the URI decoded from the image data 122 can include: a user account identifier and / or a session identifier of the session conducted on the online computing device 104 when presenting the encoded image 160 on the web page 120. The mobile computing device 106 can, for example, provide the user account / session identifier in the content request 134, and the computing server 102 can provide the web content 136 that facilitates the connection and / or continuation of the user session on the mobile computing device 106. For example, if the computing device 104 presents data of a specific website when logging in to a user account, the web content 136 can facilitate the process of logging the user account into the same website, or can automatically log the user account into the website when using the mobile computing device 106. For example, a session identifier generated for the session of the user on the online computing device 104 can be provided by the computing server 102 for use during the session of the user on the mobile computing device 106. For example, by connecting the user sessions conducted on the online computing device 104 and the mobile computing device 106, the system described in this document can improve the conversion tracking accuracy, which may be lost if the user sessions are considered in isolation.
[0038] At (234), the mobile device 206 can determine whether it is enabled for an augmented reality experience. Referring again to Figure 1, for example, a mobile computing device 106 (e.g., a web browser 132) can check the device's specifications (e.g., model, operating system, web browser, etc.) to determine whether the device, the browser, or both are augmented reality-enabled. As another example, if the mobile computing device 106 cannot locally determine whether the device / browser is augmented reality-enabled, the device 106 communicates with a web service (not shown) that can make such a determination.
[0039] For example, if the mobile device 206 is not augmented reality-enabled, the mobile device 206 can present a web page (240). Referring again to Figure 1 , for example, the web browser 132 of the mobile computing device 106 can use the web page data included in the web page content 136 to present a web page 140 without an augmented reality experience 142. In some implementations, a notification that the mobile device is not augmented reality-enabled can be provided. For example, the mobile computing device 106 can present a message on the web page 140 that the device does not enable an augmented reality experience.
[0040] For example, if the mobile device 206 is augmented reality-enabled, the mobile device 206 can present an augmented reality experience (236). In some examples, presenting the augmented reality experience can include the web browser selecting a specific augmented reality presentation from multiple augmented reality presentations of a web page. Referring again to Figure 1 , for example, during stage (E), the mobile computing device 106 can use the web browser 132 to select an augmented reality presentation that may appear on the web page 140 from multiple different augmented reality presentations, and can select an appropriate augmented reality presentation for the augmented reality experience 142. For example, to select an augmented reality presentation, the web browser 132 can execute code that parses the URI of the web page 140 and detects one or more augmented reality presentation parameters in the parsed URI. For example, based on one or more augmented reality presentation parameters, the web browser 132 can select specific augmented reality data for the augmented reality presentation identified by the parameters from the web page content 136 received from one or more computing servers 102. For example, selecting the specific augmented reality data can include selecting from multiple different models of an object according to the parameters and / or modifying a base model according to the parameters.
[0041] In some implementations, one or more augmented reality presentation parameters can include a presentation angle parameter for specifying an initial presentation angle of a three-dimensional model of an object. For example, when rendering a virtual image of an object for presentation in the augmented reality experience 142, the three-dimensional model of the object can be rotated along the X, Y, and / or Z axes according to the presentation angle parameter. For example, it can be interacted with the web page 120 (e.g., Figure 3AThe presentation angle parameter is specified by a user (e.g., by selecting one of the optional images 314) as shown. As another example, the presentation angle parameter can be specified by one or more computing servers 102 based on the presentation angle parameter used to present the virtual image of another object to the user during a previous augmented reality experience. For example, when participating in an augmented reality experience using a furniture website, the user may have previously oriented the virtual image of an object (e.g., furniture) such that the object is in a front view or depicts another specific side of the object (e.g., left side, right side, or rear). Such user preference data can be maintained by the computing server 102 in the website data storage device 110 and referenced when providing web content 136 including augmented reality data to the mobile computing device 106 to indicate that subsequent virtual objects will initially be presented in a similar orientation to the previous virtual objects.
[0042] In some implementations, one or more augmented reality presentation parameters can include a color parameter for specifying the initial presentation color of a three-dimensional model of an object. For example, when rendering the virtual image of an object for presentation in an augmented reality experience 142, the three-dimensional model of the object can be rendered in various different colors according to the presentation color parameter. For example, the presentation color can include various color attributes (e.g., hue, saturation, lightness, and other suitable color attributes), and / or can include various color patterns (e.g., roughness, bumpiness, gloss, and other suitable color patterns) that simulate materials such as leather, cloth, iron, chrome, etc. For example, the presentation color parameter can be specified by a user who interacts with the web page 120 (e.g., Figure 3A as shown by selecting one of the optional images 314). As another example, the presentation color parameter can be specified by one or more computing servers 102 based on the presentation color parameter used to present the virtual image of another object to the user during a previous augmented reality experience. For example, when participating in an augmented reality experience using a furniture website, the user may have previously specified that the virtual image of an object (e.g., furniture) be presented in brown. Such user preference data can be maintained by the computing server 102 in the website data storage 110 and referenced when providing web content 136 including augmented reality data to the mobile computing device 106 to indicate that subsequent virtual objects will initially be presented in a similar color to the previous virtual objects.
[0043] In some implementations, one or more augmented reality presentation parameters can include a size parameter for specifying the initial presentation size of a three-dimensional model of an object. For example, when rendering the virtual image of an object for presentation in an augmented reality experience 142, the three-dimensional model of the object can be rendered in various different sizes according to the presentation size parameter. For example, the size parameter can be specified by a user who interacts with the web page 120 (e.g., Figure 3AThe rendering size parameter is specified by a user (e.g., by selecting one of the optional images 314). As another example, the rendering size parameter can be specified by one or more computing servers 102 based on the rendering size parameter for presenting a virtual image of another object to the user during a previous augmented reality experience. For example, when participating in an augmented reality experience using a furniture website, the user may have previously specified that the virtual image of an object (e.g., furniture or other types of objects) be presented in a large size. Such user preference data can be maintained by the computing server in the website data storage device 110 and referenced when providing web page content 136 including augmented reality data to the mobile computing device 106 to indicate that subsequent virtual objects will initially be presented in a size similar to the previous virtual object.
[0044] In some implementations, discrete web page components can be used to perform the selection and rendering of augmented reality presentations. For example, the web page 140 ( Figure 1 shown) can include various web page components (e.g., controls such as the button 330), each implemented by a portion of code (e.g., a script) provided in the web page content 136. For example, some components can include web page buttons for selecting an image of an object on the web page (e.g., Figure 3A the selectable image 314 shown). For example, when rendering the web page 140, the web browser 132 can generate each of the page components. For example, for one of the web page components (e.g., Figure 1 shown, the corresponding web page component that has been selected on the corresponding web page 120), the web browser 132 can detect the identifier of the web page component in the URI and can select the augmented reality presentation of the web page component based on the detection of the identifier.
[0045] After selecting the augmented reality presentation, for example, the web browser 132 can determine to skip the analysis of the URIs of other web page components. For example, by using the same web page content 112, 136 to render the web pages on the corresponding devices 104, 106 and associating the augmented reality presentation with controls (e.g., buttons) for launching object-specific augmented reality experiences on the corresponding web pages 120, 140, a consistent web page codebase can be maintained while providing a consistent and flexible user experience across different devices and efficient application of user selections. This can also reduce the memory storage requirements of the computing servers 102 as they provide the same web page content 112, 136 to any device.
[0046] In some implementations, presenting an augmented reality experience can include analyzing, by discrete components of web page 140, whether to present an augmented reality experience 142. In these implementations, web page content 112, 136 can include data for only a single augmented reality experience. The web page 140 components can analyze the URI to determine which component should initiate the augmented reality experience. For example, the component that initiates the augmented reality experience 142 can affect the presentation of the augmented reality experience 142, the content presented after the augmented reality experience 142 ends, or both. For example, a component for a stereogram object facing right can initiate the augmented reality experience when the rendered object faces right, while a component for a stereogram object facing left will initiate the augmented reality experience when the rendered object faces left. For example, when the augmented reality experience for an object facing right ends, web page 140 can present the component associated with that right-facing stereogram, such as a stereogram of the object facing right. In contrast, when the augmented reality experience for an object facing left ends, web page 140 can present the component associated with that left-facing stereogram, such as a stereogram of the object facing left.
[0047] After selecting an augmented reality presentation, for example, the web browser 132 of the mobile computing device 106 can automatically present the augmented reality presentation of web page 140 and can conduct the augmented reality experience 142 (e.g., within web page 140 or another application). In some implementations, the augmented reality presentation can be automatically presented in response to determining that the mobile device is enabled to generate an augmented reality presentation. For example, after the mobile computing device 106 uses the camera device 130 to capture image data 122, the mobile computing device 106 can provide a content request 134 to the computing server 102 without user input, receive web page content 136 from the computing server, and render web page 140 with the augmented reality experience 142.
[0048] Now referring to Figure 4A , an example interface 400 for presenting an augmented reality experience is shown (e.g., similar to Figure 1 the web page 140 shown). For example, the interface 400 incorporates a virtual image 410 of an object (e.g., a chair) that is initially rendered according to one or more augmented reality presentation parameters (e.g., front angle, brown, large size) in the parsed uniform resource indicator. The augmented reality presentation in this example also includes the camera device 130 (also shown in Figure 1 ) of the mobile computing device 106 for capturing the real-world scene 420 (shown in Figure 1The data shown (in the figure). As shown in this example, the virtual image 410 of the object is integrated into the real-world scene 420, and the composite image of the image 410 and the scene 420 is presented on the interface 400. For example, the virtual image 410 can be registered within the real-world scene 420 such that changes in the scene 420 caused by a change in the position of the imaging device 130 can cause appropriate changes in the virtual image 410 of the object to reflect different perspectives from the viewpoint of the imaging device.
[0049] Referring again to Figure 2 , at (238), the mobile device 206 can receive data indicating that the augmented reality presentation should end, and in response, a web page (240) can be presented. As Figure 4A shown, for example, the interface 400 can include a control 430 through which the user can end the augmented reality presentation. In response to the user's selection of the control 430, for example, the web browser 132 of the mobile computing device 106 can end the augmented reality presentation and can present the web page 450 ( Figure 4B shown).
[0050] In some implementations, after receiving data indicating that the augmented reality presentation should end, presenting the web page can occur without the mobile device requesting additional content from the server system. For example, the web browser 132 can use the web page data included in the web page content 136 previously received by the mobile computing device 106 to present the web page 450 without requesting additional data from the computing server 102. The user can continue to interact with the web page 450, for example, by selecting another alternative image 454 of the object, and start another augmented reality experience that includes a virtual image of a model corresponding to the selected image of the object. Compared with the control 330 ( Figure 3B shown in the figure) that can be used to generate the encoded image 360 ( Figure 3A shown in the figure) to initiate an augmented reality experience of an object on another device, for example, the user's selection of the control 460 ( Figure 4B shown in the figure) can enable the user to initiate an augmented reality experience of an object on the same device. For example, the functionality of the augmented reality launch controls (e.g., 330, 460) can be determined by the web browser presenting the control and can vary based on device / browser capabilities. The user selection for the augmented reality experience can be provided by the mobile computing device 106 ( Figure 1 shown in the figure) to the computing server 102 (also shown in Figure 1 the figure), for example, to interact with the website data 110 (also shown in Figure 1It is maintained together with the user account information shown in []. For example, when initially presenting a virtual image of an object during a subsequent augmented reality experience, such user selections can be used to select and / or modify the model of the object.
[0051] The order of operations described above for process 200 is merely illustrative and can be performed in a different order. For example, the mobile computing device 206 can present a web page (240) before presenting the augmented reality experience (236). As another example, the mobile device 206 can determine whether the device is augmented reality enabled (234) before requesting a web page (226).
[0052] In some implementations, process 200 can include additional operations, fewer operations, or some of the operations can be divided into multiple sub-operations. For example, the computing device 204 can present the encoded image (220) without receiving an augmented reality control selection (218). As another example, the computing device 204 can capture the encoded image (222) without the computing device 204 presenting an image (e.g., the encoded image can be on a printed page).
[0053] For cases where the systems discussed herein collect personal information about a user or where personal information may be utilized, the user can be provided with an opportunity to control whether the program or feature collects personal information (e.g., information about the user's web activities or user preferences), or to control whether and / or how content that may be more relevant to the user is received from a content server. Additionally, certain data can be anonymized in one or more ways before it is stored or used so that personally identifiable information is removed. For example, the identity of a user can be anonymized so that the user's personally identifiable information cannot be determined, or the user's geographical location can be generalized to a location where location information (such as a city, zip code, or state) is obtained so that the user's specific location cannot be determined. Thus, the user can control how information about him or her is collected and how that information is used by the content server.
[0054] Embodiments of the subject matter and the functional operations described in this specification can be implemented in digital electronic circuitry, in tangibly embodied computer software or firmware, in computer hardware including the structures disclosed in this specification and their structural equivalents, or in combinations of one or more of them. Embodiments of the subject matter described in this specification can be implemented as one or more computer programs, i.e., one or more modules of computer program instructions encoded on a tangible non-transitory program carrier, to be executed by, or to control the operation of, a data processing apparatus. Alternatively or additionally, the program instructions can be encoded on an artificially generated propagated signal, e.g., a machine-generated electrical, optical, or electromagnetic signal, that is generated to encode information for transmission to the appropriate receiver apparatus for execution by the data processing apparatus. A computer storage medium may be a machine-readable storage device, a machine-readable storage substrate, a random or serial access storage device, or a combination of one or more of them.
[0055] The term “data processing apparatus” refers to data processing hardware and includes various devices, apparatuses, and machines for processing data, such as including programmable processors, computers, or multiple processors or computers. The apparatus may also be, or further include, special purpose logic circuitry, such as an FPGA (field programmable gate array) or an ASIC (application specific integrated circuit). In addition to hardware, the apparatus may optionally include code that creates an execution environment for the computer program, e.g., code that constitutes processor firmware, a protocol stack, a database management system, an operating system, or a combination of one or more of them.
[0056] A computer program (also referred to as or described as a program, software, software application, module, software module, script, or code) can be written in any form of programming language, including a compiled or interpreted language, or a declarative or procedural language, and can be deployed in any form, including as a stand-alone program or as a module, component, subroutine, or other unit suitable for use in a computing environment. A computer program may, but need not, correspond to a file in a file system. The program may be stored in a portion of a file that holds other programs or data (e.g., one or more scripts stored in a markup language document), in a single file dedicated to the program being discussed, or in multiple collaborating files (e.g., files that store one or more modules, subroutines, or portions of code). A computer program may be deployed to execute on one computer or on multiple computers located at one site or distributed across multiple sites and interconnected by a communication network.
[0057] The processes and logical flows described in this specification can be performed by one or more programmable computers executing one or more computer programs to perform functions by operating on input data and generating output. The processes and logical flows can also be performed by, and apparatus can also be implemented as, special purpose logic circuitry, such as an FPGA (Field Programmable Gate Array) or an ASIC (Application Specific Integrated Circuit).
[0058] For example, computers suitable for executing computer programs include general or special purpose microprocessors, or both, or any other kind of central processing unit. Generally, a central processing unit will receive instructions and data from a read only memory or a random access memory, or both. Fundamental elements of a computer are a central processing unit for executing or running instructions, and one or more memory devices for storing instructions and data. Generally, a computer will also include, or be operatively coupled to, one or more mass storage devices, such as magnetic disks, magneto-optical disks, or optical disks, for storing data, from which it can receive data or to which it can transfer data, or both. However, a computer need not have such devices. In addition, a computer may be embedded in another device, such as a mobile telephone, a smart phone, a personal digital assistant (PDA), a mobile audio or video player, a game console, a global positioning system (GPS) receiver, or a portable storage device, such as a universal serial bus (USB) flash drive, to name just a few.
[0059] Computer-readable media suitable for storing computer program instructions and data include all forms of non-volatile memory, media and storage devices. By way of example, and not limitation, semiconductor storage devices include, for example, EPROM, EEPROM, and flash memory devices; magnetic disks, such as internal hard disks or removable disks; magneto-optical disks; and CD-ROM and DVD-ROM disks. The processor and the memory can be supplemented by, or incorporated in, special purpose logic circuitry.
[0060] To provide for interaction with a user, implementations of the subject matter described in this specification can be implemented on a computer having: a display device for displaying information to the user, such as an LCD (Liquid Crystal Display), an OLED (Organic Light Emitting Diode), or other monitor; and a keyboard and a pointing device by which the user can provide input to the computer, such as a mouse or a trackball. Other types of devices can also be used to provide for interaction with the user: for example, the feedback provided to the user can be any form of sensory feedback, such as, visual feedback, auditory feedback, or tactile feedback; and input from the user can be received in any form including auditory, speech, or tactile input. Additionally, the computer can interact with the user by sending documents to and receiving documents from the device used by the user, for example, by sending a web page to a web browser on a client device of the user in response to a request received from the web browser.
[0061] Implementations of the subject matter described in this specification can be implemented in a computing system that includes a backend component, such as a data server; or includes a middleware component, such as an application server; or includes a frontend component, such as a client computer having a graphical user interface or a web browser, by which the user can interact with an implementation of the subject matter described in this specification; or any combination of one or more such backend, middleware, or frontend components. The components of the system can be interconnected by any form or medium of digital data communication such as a communication network. Examples of communication networks include a local area network (LAN) and a wide area network (WAN) such as the Internet.
[0062] The computing system can include a client and a server. The client and the server are typically located remotely from each other and typically interact via a communication network. The relationship of the client and the server arises by virtue of computer programs running on respective computers and having a client-server relationship to each other. In some implementations, the server sends data such as a hypertext markup language (HTML) page to a user device, for example, in order to display data to a user interacting with the user device acting as a client and to receive user input from the user interacting with the user device. Data generated at the user device, such as results of user interaction, can be received at the server from the user device.
[0063] Figure 5Block diagrams of computing devices 500, 550, which can be used as a client or as a server or multiple servers, can be used to implement the systems and methods described in this document. Computing device 500 is intended to represent various forms of digital computers, such as laptop computers, desktop computers, workstations, personal digital assistants, servers, blade servers, mainframes, and other suitable computers. Computing device 550 is intended to represent various forms of mobile devices, such as personal digital assistants, cellular phones, smartphones, smartwatches, head-mounted devices, and other similar computing devices. The components shown herein, their connections and relationships, and their functions are only intended to be exemplary and are not intended to limit the implementations described and / or claimed in this document.
[0064] Computing device 500 includes: a processor 502, a memory 504, a storage device 506, a high-speed interface 508 connected to the memory 504 and a high-speed expansion port 510, and a low-speed interface 512 connected to a low-speed bus 514 and the storage device 506. Each of the components 502, 504, 506, 508, 510, and 512 is interconnected using various buses and can be suitably mounted on a common motherboard or otherwise. Processor 502 can process instructions for execution within computing device 500, including instructions stored in memory 504 or on storage device 506, to display graphical information of a GUI on an external input / output device such as a display 516 coupled to the high-speed interface 508. In other implementations, multiple processors and / or multiple buses can be suitably used with multiple memories and multiple types of memory. Additionally, multiple computing devices 500 can be connected to each device that provides a portion of the necessary operations (e.g., a server group, a set of blade servers, or a multiprocessor system).
[0065] Memory 504 stores information within computing device 500. In one implementation, memory 504 is a computer-readable medium. In one implementation, memory 504 is one or more volatile memory units. In another implementation, memory 504 is one or more non-volatile memory units.
[0066] The storage device 506 can provide mass storage for the computing device 500. In one implementation, the storage device 506 is a computer-readable medium. In various different implementations, the storage device 506 can be a floppy disk device, a hard disk device, an optical disk device, or a magnetic tape device, a flash memory, or other similar solid-state storage devices, or an array of devices including devices in a storage area network or other configurations. In one implementation, the computer program product is tangibly embodied in an information carrier. The computer program product contains the following instructions, which, when executed, perform one or more methods such as those described above. The information carrier is a computer-readable medium or a machine-readable medium, such as the memory 504, the storage device 506, or the memory on the processor 502.
[0067] The high-speed controller 508 manages the bandwidth-intensive operations of the computing device 500, while the low-speed controller 512 manages the less bandwidth-intensive operations. Such a division of responsibilities is merely exemplary. In one implementation, the high-speed controller 508 is coupled to the memory 504, the display 516 (e.g., via a graphics processor or accelerator), and is coupled to the high-speed expansion port 510, which can accommodate various expansion cards (not shown). In an implementation, the low-speed controller 512 is coupled to the storage device 506 and the low-speed expansion port 514. The low-speed expansion port, which can include various communication ports (e.g., USB, Bluetooth, Ethernet, wireless Ethernet), can be coupled to one or more input / output devices, such as a keyboard, a pointing device, a scanner, or a networking device such as a switch or a router, for example, via a network adapter.
[0068] As shown in the accompanying drawings, the computing device 500 can be implemented in a variety of different forms. For example, the computing device 500 can be implemented as a standard server 520, or be implemented multiple times in a group of such servers. The computing device 500 can also be implemented as part of a rack-mounted server system 524. Additionally, the computing device 500 can be implemented in a personal computer such as a laptop computer 522. Alternatively, the components from the computing device 500 can be combined with other components in a mobile device (not shown) such as the device 550. Each of such devices can contain one or more of the computing devices 500, 550, and the entire system can consist of multiple computing devices 500, 550 that communicate with each other.
[0069] The computing device 550 includes a processor 552, a memory 564, input / output devices such as a display 554, a communication interface 566, a transceiver 568, and other components. The device 550 may also be provided with a storage device such as a microdrive or other device to provide additional storage. Each of the components 550, 552, 564, 554, 566, and 568 is interconnected using various buses, and several of the components may be appropriately mounted on a common motherboard or otherwise installed.
[0070] The processor 552 may process instructions for execution within the computing device 550, including instructions stored in the memory 564. The processor may also include separate analog and digital processors. The processor may, for example, provide the coordination of the other components of the device 550, such as the control of the user interface, applications running on the device 550, and wireless communications through the device 550.
[0071] The processor 552 may communicate with a user through a control interface 558 and a display interface 556 coupled to the display 554. The display 554 may be, for example, a TFT LCD display or an OLED display, or other suitable display technology. The display interface 556 may include appropriate circuitry for driving the display 554 to present graphics and other information to the user. The control interface 558 may receive commands from the user and convert them for submission to the processor 552. Additionally, an external interface 562 may be provided to communicate with the processor 552 to enable the device 550 to communicate with other devices in a near field. The external interface 562 may provide, for example, wired communication (e.g., via a docking process) or wireless communication (e.g., via Bluetooth or other such technology).
[0072] The memory 564 stores information within the computing device 550. In one implementation, the memory 564 is a computer-readable medium. In one implementation, the memory 564 is one or more volatile memory units. In another implementation, the memory 564 is one or more non-volatile memory units. An extended memory 574 may also be provided and the extended memory 574 is connected to the device 550 through an extended interface 572, which may include, for example, a SIMM card interface. Such an extended memory 574 may provide additional storage space for the device 550, or may also store applications or other information for the device 550. Specifically, the extended memory 574 may include instructions for performing or supplementing the above processes, and may also include security information. Thus, for example, the extended memory 574 may be provided as a security module for the device 550 and may be programmed with instructions that allow for the secure use of the device 550. Additionally, security applications and additional information may be provided via the SIMM card, such as placing identification information on the SIMM card in a way that cannot be hacked.
[0073] The memory may include, for example, flash memory and / or MRAM memory, as described below. In one implementation, a computer program product is tangibly embodied in an information carrier. The computer program product contains instructions that, when executed, perform one or more methods such as those described above. The information carrier is a computer-readable medium or a machine-readable medium, such as the memory 564, the extended memory 574, or the memory on the processor 552.
[0074] The device 550 may communicate wirelessly via the communication interface 566, which may include digital signal processing circuitry when necessary. The communication interface 566 may provide communication in various modes or protocols (such as GSM voice calls, SMS, EMS, or MMS messaging, CDMA, TDMA, PDC, WCDMA, CDMA 2000, or GPRS, etc.). Such communication may occur, for example, via the radio frequency transceiver 568. Additionally, short-range communication may occur, such as using Bluetooth, WiFi, or other such transceivers (not shown). Additionally, the GPS receiver module 570 may provide additional wireless data to the device 550, which may be appropriately used by applications running on the device 550.
[0075] The device 550 may also communicate audibly using the audio codec 560, which may receive voice information from a user and convert it into usable digital information. The audio codec 560 may also generate audible sounds for the user, for example, via a speaker in the handset of the device 550. Such sounds may include sounds from a voice telephone call, may include recorded sounds (such as voice messages, music files, etc.), and may also include sounds generated by applications operating on the device 550.
[0076] As shown in the drawings, the computing device 550 may be implemented in a variety of different forms. For example, the computing device 550 may be implemented as a cellular phone 580. The computing device 550 may also be implemented as part of a smart phone 582, a personal digital assistant, or other similar mobile devices.
[0077] The various implementations of the systems and techniques described herein can be implemented in digital electronic circuitry, integrated circuitry, specially designed ASICs (application specific integrated circuits), computer hardware, firmware, software, and / or combinations thereof. These various implementations can include implementations in one or more computer programs, which can be executed and / or interpreted on a programmable system including at least one programmable processor, which can be special purpose or general purpose, coupled to receive data and instructions from, and to transmit data and instructions to, a storage system, at least one input device, and at least one output device.
[0078] These computer programs (also referred to as programs, software, software applications, or code) include machine instructions for a programmable processor and can be implemented in high-level procedural and / or object-oriented programming languages and / or in assembly language / machine language. As used herein, the term "machine-readable medium," "computer-readable medium" refers to any computer program product, apparatus, and / or device (e.g., a disk, optical disk, memory, programmable logic device (PLD)) used to provide machine instructions and / or data to a programmable processor, including a machine-readable medium that receives machine instructions as a machine-readable signal. The term machine-readable signal refers to any signal used to provide machine instructions and / or data to a programmable processor.
[0079] Although this specification contains many specific implementation details, these should not be construed as limitations on the scope that can be claimed, but rather as descriptions of features specific to particular implementations. Certain features described in the context of separate implementations in this specification can also be implemented in combination in a single implementation. Conversely, the various features described in the context of a single implementation can also be implemented separately or in any suitable sub-combination in multiple implementations. In addition, although features may be described above as acting in certain combinations and even initially claimed as such, one or more features from a claimed combination can in some cases be excluded from the combination, and the claimed combination can cover a sub-combination or a variation of a sub-combination.
[0080] Similarly, although the operations are depicted in the drawings in a particular order, this should not be construed as requiring that the operations be performed in the particular order shown or in a sequential order, or that all of the illustrated operations be performed to achieve the desired result. In some cases, multitasking and parallel processing may be advantageous. Additionally, the separation of various system modules and components in the above-described implementations should not be construed as requiring such separation in all implementations, and it should be understood that the described program components and systems can generally be integrated together in a single software product or packaged into multiple software products.
[0081] Particular embodiments of the subject matter have been described. Other embodiments are within the scope of the following claims. For example, the acts recited in the claims can be performed in a different order and still achieve the desired result. As one example, the processes depicted in the figures need not be in the particular order shown or in sequential order to achieve the desired result. In some cases, multitasking and parallel processing may be advantageous.
Claims
1. A computer-implemented method, comprising: providing, by a mobile device, a content request for a web page identified by a uniform resource identifier to a computer; receiving, in response to providing the content request for the web page identified by the uniform resource identifier, data of the web page identified by the uniform resource identifier by a mobile device web browser running on the mobile device, the data of the web page including augmented reality data for a plurality of augmented reality presentations; selecting, by the mobile device web browser, an augmented reality presentation from the plurality of augmented reality presentations of the web page, the selection comprising: parsing the uniform resource identifier identifying the web page; detecting one or more augmented reality presentation parameters in the parsed uniform resource identifier; and using the one or more augmented reality presentation parameters detected in the parsed uniform resource identifier, selecting, from the plurality of augmented reality presentations, the augmented reality presentation identified by the one or more augmented reality presentation parameters and associated with an image included in the web page, and skipping selecting a second augmented reality presentation from the plurality of augmented reality presentations that is not identified by the one or more augmented reality presentation parameters; automatically presenting, by the mobile device web browser, the augmented reality presentation identified by the one or more augmented reality presentation parameters and data from a camera device included in the mobile device on which the mobile device web browser is running, and skipping presenting the second augmented reality presentation from the plurality of augmented reality presentations that is not identified by the one or more augmented reality presentation parameters; after automatically presenting the augmented reality presentation, receiving, by the mobile device web browser, data indicating that the augmented reality presentation should end; and after receiving the data indicating that the augmented reality presentation should end, presenting, by the mobile device web browser, the web page using the data of the web page.
2. The computer-implemented method according to claim 1, wherein, Receiving the data of the web page includes: receiving data independent of the underlying operating system of the mobile device.
3. The computer-implemented method according to claim 1, wherein, Receiving the data of the web page including the augmented reality data includes: receiving the augmented reality data specific to the mobile device, an account of a user of the mobile device, or both.
4. The computer-implemented method according to claim 1, wherein, Receiving the data of the web page including the augmented reality data includes: receiving the augmented reality data including a three-dimensional model of an object.
5. The computer-implemented method according to claim 4, wherein, Detecting the one or more augmented reality presentation parameters in the parsed uniform resource identifier includes: detecting a presentation angle parameter for specifying an initial presentation angle of the three-dimensional model of the object.
6. The computer-implemented method according to claim 4, wherein, Detecting the one or more augmented reality presentation parameters in the parsed uniform resource identifier includes: detecting a color parameter for specifying an initial presentation color of the three-dimensional model of the object.
7. The computer-implemented method according to claim 4, wherein, Detecting the one or more augmented reality presentation parameters in the parsed uniform resource identifier includes: detecting a size parameter for specifying an initial presentation size of the three-dimensional model of the object.
8. The computer-implemented method according to claim 1, further comprising: Receiving, from a camera device of the mobile device, image data encoding the uniform resource identifier; Determining, by the mobile device, the uniform resource identifier based on the image data; 9. The computer-implemented method according to claim 8, wherein, Determining the image data based on an image visually presented by a computing device different from the mobile device and captured by the camera device of the mobile device; 10. The computer-implemented method according to claim 9, wherein, The uniform resource identifier encoded in the image data visually presented by the computing device includes a session identifier of a session between the computing device and a server, and the mobile device web browser uses the session identifier to associate a first user session between the computing device and the server with a second user session between the mobile device and the server; 11. The computer-implemented method according to claim 1, wherein, Automatically presenting the augmented reality presentation in response to determining that the mobile device is enabled to generate the augmented reality presentation; 12. The computer-implemented method according to claim 1, wherein, After receiving data indicating that the augmented reality presentation should end, presenting, by the mobile device web browser, the web page occurs without the mobile device requesting additional content from the server system; 13. The computer-implemented method according to claim 1, further comprising: For each of two or more web page components related to content depicted in the web page, generating, by the mobile device web browser, the web page components to cause presentation of the related content; For a first web page component of the two or more web page components, detecting an identifier of the first web page component in the uniform resource identifier, wherein: Selecting an augmented reality presentation identified by the one or more augmented reality presentation parameters and associated with an image included in the web page includes: In response to detecting the identifier of the first web page component in the uniform resource identifier, using the detected identifier of the first web page component in the uniform resource identifier to select an augmented reality presentation of the first web page component; and Skipping selecting a second augmented reality presentation not identified by the one or more augmented reality presentation parameters from the plurality of augmented reality presentations includes: In response to detecting the identifier of the first web page component in the uniform resource identifier, for a second web page component of the two or more web page components, determining to skip analysis of the uniform resource identifier in response to detection of the identifier of the first web page component; 14. A system comprising: One or more computers and one or more storage devices, on which instructions are stored, the instructions, when executed by the one or more computers, being operative to cause the one or more computers to perform operations including the following steps: Providing, by a mobile device, a content request for a web page identified by a uniform resource identifier to a computer; In response to providing the content request for the web page identified by the uniform resource identifier, receiving, by a mobile device web browser running on the mobile device, data of the web page identified by the uniform resource identifier, the data of the web page including augmented reality data for a plurality of augmented reality presentations; Selecting, by the mobile device web browser, an augmented reality presentation from the plurality of augmented reality presentations of the web page, the selection including: Parsing the uniform resource identifier that identifies the web page; Detecting one or more augmented reality presentation parameters in the parsed uniform resource identifier; and Using the one or more augmented reality presentation parameters detected in the parsed uniform resource identifier, selecting, from the plurality of augmented reality presentations, the augmented reality presentation identified by the one or more augmented reality presentation parameters and associated with an image included in the web page, and skipping selecting a second augmented reality presentation from the plurality of augmented reality presentations that is not identified by the one or more augmented reality presentation parameters; Automatically presenting, by the mobile device web browser, the augmented reality presentation identified by the one or more augmented reality presentation parameters and data from a camera device included in the mobile device on which the mobile device web browser is running, and skipping presenting the second augmented reality presentation from the plurality of augmented reality presentations that is not identified by the one or more augmented reality presentation parameters; After automatically presenting the augmented reality presentation, receiving, by the mobile device web browser, data indicating that the augmented reality presentation should end; and After receiving the data indicating that the augmented reality presentation should end, presenting, by the mobile device web browser, the web page using the data of the web page.
15. The system according to claim 14, the operation further including: Receiving, from the camera device of the mobile device, image data encoding the uniform resource identifier; Determining, by the mobile device, the uniform resource identifier based on the image data.
16. The system according to claim 14, wherein, Automatically presenting the augmented reality presentation in response to determining that the mobile device is enabled to generate the augmented reality presentation.
17. The system according to claim 14, wherein, After receiving the data indicating that the augmented reality presentation should end, presenting, by the mobile device web browser, the web page in the event that the mobile device does not request additional content from the server system.
18. A non-transitory computer storage medium encoded with instructions that, when executed by one or more computers, cause the one or more computers to perform an operation including the following steps: Providing, by a mobile device, a content request for a web page identified by a uniform resource identifier to a computer; In response to providing the content request for the web page identified by the uniform resource identifier, receiving, by a mobile device web browser running on the mobile device, data of the web page identified by the uniform resource identifier, the data of the web page including augmented reality data for a plurality of augmented reality presentations; Selecting, by the mobile device web browser, an augmented reality presentation from the plurality of augmented reality presentations of the web page, the selection including: Parsing the uniform resource identifier that identifies the web page; Detecting one or more augmented reality presentation parameters in the parsed uniform resource identifier; and Select, using the one or more augmented reality rendering parameters detected in the parsed uniform resource identifier, an augmented reality rendering identified by the one or more augmented reality rendering parameters and associated with an image included in the web page from the plurality of augmented reality renderings, and skip selecting a second augmented reality rendering from the plurality of augmented reality renderings that is not identified by the one or more augmented reality rendering parameters; Automatically render, by the mobile device web browser, the augmented reality rendering identified by the one or more augmented reality rendering parameters and data from a camera device included in the mobile device on which the mobile device web browser is running, and skip rendering the second augmented reality rendering from the plurality of augmented reality renderings that is not identified by the one or more augmented reality rendering parameters; After automatically rendering the augmented reality rendering, receive, by the mobile device web browser, data indicating that the augmented reality rendering should end; and After receiving the data indicating that the augmented reality rendering should end, render, by the mobile device web browser, the web page using the data of the web page.
19. The non-transitory computer storage medium according to claim 18, the operation further comprising: Receive, from a camera device of the mobile device, image data encoding the uniform resource identifier; Determine, by the mobile device, the uniform resource identifier based on the image data.
20. The non-transitory computer storage medium according to claim 18, wherein, Automatically render the augmented reality rendering in response to determining that the mobile device is enabled to generate the augmented reality rendering.
21. The non-transitory computer storage medium according to claim 18, wherein, After receiving the data indicating that the augmented reality rendering should end, render, by the mobile device web browser, the web page occurs where the mobile device does not request additional content from the server system.
Citation Information
Patent Citations
Apparatus and method providing augmented reality contents based on web information structure
US20140289607A1
System and method for providing augmented reality on mobile devices
US20150348329A1
Systems and Methods for Augmented Reality Aware Contents
US20180040166A1
Teleportation Links for Mixed Reality Environments
US20180059902A1