84 results about "Cross reality" patented technology

A cross reality system enables any of multiple devices to efficiently and accurately access previously stored maps and render virtual content specified in relation to those maps. Both stored maps and tracking maps used by portable devices may have wireless fingerprints associated with them. The portable devices may maintain wireless fingerprints based on wireless scans performed repetitively, based on one or more trigger conditions, as the devices move around the physical world. The wireless information obtained from these scans may be used to create or update wireless fingerprints associated with locations in a tracking map on the devices. One or more of these wireless fingerprints may be used when a previously stored map is to be selected based on its coverage of an area in which the portable device is operating. Maintaining wireless fingerprints in this way provides a reliable and low latency mechanism for performing map-related operations.

A cross reality system enables any of multiple devices to efficiently render shared location-based content. The cross reality system may include a cloud-based service that responds to requests from devices to localize with respect to a stored map. The service may return to the device information that localizes the device with respect to the stored map. In conjunction with localization information, the service may provide information about locations in the physical world proximate the device for which virtual content has been provided. Based on information received from the service, the device may render, or stop rendering, virtual content to each of multiple users based on the user's location and specified locations for the virtual content.

A cross reality system that provides an immersive user experience shared by multiple user devices by providing quality information about a shared map. The quality information may be specific to individual user devices rendering virtual content specified with respect to the shared map. The quality information may be provided for persistent coordinate frames (PCFs) in the map. The quality information about a PCF may indicate positional uncertainty of virtual content, specified with respect to the PCF, when rendered on the user device. The quality information may be computed as upper bounding errors by determining error statistics for one or more steps in a process of specifying position with respect to the PCF or transforming that positional expression to a coordinate frame local to the device for rendering the virtual content. Applications running on individual user devices may adjust the rendering of virtual content based on the quality information about the shared map.

A cross reality system enables any of multiple devices to efficiently and accurately access previously stored maps and render virtual content specified in relation to those maps. Both stored maps and tracking maps used by portable devices may have wireless fingerprints associated with them. The portable devices may maintain wireless fingerprints based on wireless scans performed repetitively, based on one or more trigger conditions, as the devices move around the physical world. The wireless information obtained from these scans may be used to create or update wireless fingerprints associated with locations in a tracking map on the devices. One or more of these wireless fingerprints may be used when a previously stored map is to be selected based on its coverage of an area in which the portable device is operating. Maintaining wireless fingerprints in this way provides a reliable and low latency mechanism for performing map-related operations.

A cross reality system enables any of multiple types of devices to efficiently and accurately access previously stored maps and render virtual content specified in relation to those maps. The cross reality system may include a cloud-based localization service that responds to requests from devices to localize with respect to a stored map. Devices of any type, with native hardware and software configured for augmented reality operations may be configured to work with the cross reality system by incorporating components that interface between the native AR framework of the device and the cloud-based localization service. These components may present position information about the device in a format recognized by the localization service. Additionally, these components may filter or otherwise process perception data provided by the native AR framework to increase the accuracy of localization.

A wearable display system with a limited number of cameras. Two cameras can be arranged to provide an overlapping central view field and a peripheral view field associated with one of the two cameras. A third camera can be arranged to provide a color view field overlapping the central view field. The wearable display system may be coupled to a processor configured to generate a world model and track hand motion in the central view field using the two cameras. The processor may be configured to perform a calibration routine to compensate for distortions during use of the wearable display system. The processor may be configured to identify and address portions of the world model including incomplete depth information by obtaining additional depth information, such as by enabling emitters, detecting planar surfaces in the physical world, or identifying relevant object templates in the world model.

A cross reality system that renders virtual content generated by executing native mode applications may be configured to render web-based content using components that render content from native applications. The system may include a Prism manager that provides Prisms in which content from executing native applications is rendered. For rendering web based content, a browser, accessing the web based content, may be associated with a Prism and may render content into its associated Prism, creating the same immersive experience for the user as when content is generated by a native application. The user may access the web application from the same program launcher menu as native applications. The system may have tools that enable a user to access these capabilities, including by creating for a web location an installable entity that, when processed by the system, results in an icon for the web content in a program launcher menu.

A portable electronic system receives a set of one or more canonical maps and determines the sparse map based at least in part upon one or more anchors pertaining to the physical environment. The sparse map is localized to at least one canonical map in the set of one or more canonical maps, and a new canonical map is created at least by merging sparse map data of the sparse map into the at least one canonical map. The set of one or more canonical maps may be determined from a universe of canonical maps comprising a plurality of canonical maps by applying a hierarchical filtering scheme to the universe. The sparse map may be localized to the at least one canonical map at least by splitting the sparse map into a plurality of connected components and by one or more merger operations.

To provide an information processing apparatus, a method for controlling an information processing apparatus, and a program with which utilization of an input device can be promoted which is used for operations in cross reality (XR).SOLUTION: An information processing apparatus 101 comprises: a hand-switch detection unit 301 that, when a user holds an input device 102 allowing the user to perform operations in XR by one hand, detects a motion of the user in changing a holding hand that is the hand holding the input device 102, as a hand-switch motion; and a holding hand discrimination unit 302 that discriminates the holding hand on the basis of the passing motion detected by the passing detection unit 301.SELECTED DRAWING: Figure 3

An augmented reality viewing system is described. A local coordinate framework of the local content will be transformed into a world coordinate framework. The head coordinate frame is further transformed, and the camera coordinate frame including all pupil positions of the eye is further transformed. One or more users may interact with the viewing system in a separate session. If the canonical map is available, the earlier map is downloaded onto the user's viewing device. The viewing device then generates another map and locates the subsequent map to the canonical map.

Provided is a technology that is for using cross reality (XR) to teach a robot while verifying the effects of a light source. A device (100) comprises an XR screen generation unit that is for generating an XR screen that blends a real space and a virtual space, an XR event unit that is for detecting events related to the XR screen (12), a placement unit that is for updating the position of a light source (146) that is provided to the tip of a robot (150) that is in the virtual space, and a teaching point storage unit that is for storing teaching points for the robot (150). The XR screen generation unit displays a workpiece and the light source (146) that are in the virtual space on the XR screen (12). The placement unit moves the light source (146) on the XR screen (12) on the basis of detection of a movement event for the light source (146). The teaching point storage unit stores teaching points for the robot (150) as moving in association with the light source (146).

Various techniques pertaining to methods, systems, and computer program products a spatial persistence process that places a virtual object relative to a physical object for an extended-reality display device based at least in part upon a persistent coordinate frame (PCF). A determination is made to decide whether a drift is detected for the virtual object relative to the physical object, upon or after detection of the drift or deviation, the drift or deviation is corrected at least by updating a tracking map into an updated tracking map and further at least by updating the persistent coordinate frame (PCF) based at least in part upon the updated tracking map, wherein the persistent coordinate frame (PCF) comprises six degrees of freedom relative to the map coordinate system.

The present technology pertains to an information processing apparatus, an information processing method, and an input device that enable a contact state of a virtual object in an XR (cross reality) space to be presented to a user. The information processing apparatus includes: an identification unit configured to identify a state of each virtual object in an XR space; a space control unit for calculating a contact condition between the virtual objects in the XR space based on the state of each virtual object and the design data of each virtual object; and a presentation control unit for controlling presentation of contact condition feedback indicating a contact condition between the virtual objects. This technology is applicable to, for example, an XR system.

A cross reality system enables any of multiple devices to efficiently and accurately access previously stored maps and render virtual content specified in relation to those maps. The cross reality system may include a cloud-based localization service that responds to requests from devices to localize with respect to a stored map. The request may include one or more sets of feature descriptors extracted from an image of the physical world around the device. Those features may be posed relative to a coordinate frame used by the local device. The localization service may identify one or more stored maps with a matching set of features. Based on a transformation required to align the features from the device with the matching set of features, the localization service may compute and return to the device a transformation to relate its local coordinate frame to a coordinate frame of the stored map.

The design article in this case is an image for a calibration operation used to calibrate each user's position information, position information and / or time information for displaying content in a space formed by images provided by XR (cross reality) in a simple, low-cost, functional, quick, and immersive manner. When each user viewing images related to the space provided by XR wears a head-mounted display capable of displaying images and places their hand in a position corresponding to a pattern displayed on a screen installed in any location, the head-mounted display detects the placed hand and executes processing to calibrate each user's position information, position information and / or time information for displaying content in the space provided by XR.

An information processing apparatus capable of promoting utilization of an input device used for operations in the XR. The information processing apparatus includes a memory device that stores a set of instructions, and at least one processor that executes the set of instructions to: detect an action of a user to change a holding hand that holds an input device as a re-holding action when the user holds the input device that allows a one-hand operation in cross reality, and determine the holding hand based on the re-holding action detected.

This document describes scene understanding for cross reality systems using occupancy grids. In one aspect, a method includes recognizing one or more objects in a model of a physical environment generated using images of the physical environment. For each object, a bounding box is fit around the object. An occupancy grid that includes a multiple cells is generated within the bounding box around the object. A value is assigned to each cell of the occupancy grid based on whether the cell includes a portion of the object. An object representation that includes information describing the occupancy grid for the object is generated. The object representations are sent to one or more devices.

This product is a head-mounted communication terminal that is worn on the head as shown in the "Reference Diagram Showing Usage," and projects images and other data onto the user's retina from a light source. This product can be used for various purposes, such as information verification and communication within XR (Cross Reality) spaces, which is a technology that merges the real world and virtual world to make non-existent things perceptible.

A control apparatus presents a three-dimensional operation to a worker using cross reality. The control apparatus is configured with a computer including a calculation device that performs a predetermined calculation process and a storage device that the calculation device is able to access. The calculation device includes an acquisition unit that acquires work information of work performed by the worker. The calculation device includes a calculation unit that performs calculation to present consecutive instruction content corresponding to the work performed by the worker to the worker. The calculation device includes an output unit that outputs content. The calculation unit includes a control unit that dynamically controls a display method for the consecutive instruction content according to a spatial divergence between the work information acquired by the acquisition unit and the consecutive instruction content.

A portable electronic system receives a set of one or more canonical maps and determines the sparse map based at least in part upon one or more anchors pertaining to the physical environment. The sparse map is localized to at least one canonical map in the set of one or more canonical maps, and a new canonical map is created at least by merging sparse map data of the sparse map into the at least one canonical map. The set of one or more canonical maps may be determined from a universe of canonical maps comprising a plurality of canonical maps by applying a hierarchical filtering scheme to the universe. The sparse map may be localized to the at least one canonical map at least by splitting the sparse map into a plurality of connected components and by one or more merger operations.

The present technology relates to a control system and a control method that make it possible to prevent detraction from a sense of immersion. Control is carried out with respect to display of an assistance guide for assisting spatial recognition which is performed by a user in a real space and performed in a virtual space displayed on a real object that is an object in the real space, such control being in accordance with the state of the user or the state of content reproduced in the virtual space. The present technology can be applied to, for example, an XR (cross reality) experience providing system that provides an XR experience.

This product is a head-mounted communication information terminal that is worn on the head, as shown in "Reference Figure 1 showing the state of use," and projects images and other data onto the user's retina from a light source. This product can be used for various purposes, such as information verification and communication in XR (cross-reality) spaces, which is a technology that merges the real world and the virtual world to make things that do not exist in reality perceptible. As shown in "Front view of the changed state," "Left side view of the changed state," "Front side perspective view of the changed state," and "Reference Figure 1 showing the state of use" and "Reference Figure 2 showing the state of use," the entire frame can be moved up and down depending on whether projection is required or not.

A cross-reality system that provides an immersive user experience by storing persistent spatial information about the physical world, where one or more user devices can access the persistent spatial information to determine their location in the physical world, and applications can access the persistent spatial information to specify the location of virtual objects in the physical world. The persistent spatial information enables users to have shared virtual and physical experiences when interacting with the cross-reality system. Additionally, the persistent spatial information can be used in a map of the physical world, enabling one or more devices to access and locate in a previously stored map, reducing the need to map the physical space before using the cross-reality system therein. The persistent spatial information can be stored as a persistent coordinate frame, which can include a transformation relative to a reference orientation and information derived from an image at a location corresponding to the persistent coordinate frame.

A cross reality system enables any of multiple devices to efficiently access previously stored maps. Both stored maps and tracking maps used by portable devices may have any of multiple types of location metadata associated with them. The location metadata may be used to select a set of candidate maps for operations, such as localization or map merge, that involve finding a match between a location defined by location information from a portable device and any of a number of previously stored maps. The types of location metadata may prioritized for use in selecting the subset. To aid in selection of candidate maps, a universe of stored maps may be indexed based on geo-location information. A cross reality platform may update that index as it interacts with devices that supply geo-location information in connection with location information and may propagate that geo-location information to devices that do not supply it.

A cross reality system enables any of multiple devices to efficiently and accurately access previously persisted maps, even maps of very large environments, and render virtual content specified in relation to those maps. The cross reality system may quickly process a batch of images acquired with a portable device to determine whether there is sufficient consistency across the batch in the computed localization. Processing on at least one image from the batch may determine a rough localization of the device to the map. This rough localization result may be used in a refined localization process for the image for which it was generated. The rough localization result may also be selectively propagated to a refined localization process for other images in the batch, enabling rough localization processing to be skipped for the other images.

To provide a cross reality system with localization service.SOLUTION: A cross reality system enables any of multiple devices to efficiently and accurately access previously stored maps and render a virtual content specified in relation to those maps. The cross reality system may include a cloud-based localization service that responds to requests from devices to localize with respect to a stored map. The request may include one or more sets of feature descriptors extracted from an image of a physical world around the device. Those features may be posed relative to a coordinate frame used by a local device. The localization service may identify one or more stored maps with a matching set of features.SELECTED DRAWING: Figure 61

A video output method produces, in an augmented reality space provided by augmenting a real space, a display about one space in a facility including a plurality of spaces in the real space, and includes: acquiring status information indicating a status of each of the plurality of spaces; identifying, among the plurality of spaces in the real space, a first space where a user is currently present; determining, based on the status information of each of the plurality of spaces, as a second space to which the first space is to be coupled, a space that is currently in a predetermined status among the plurality of spaces in the real space; and outputting presentation information for presenting, to the user, via a cross reality (XR) device worn by the user, a video in which the second space determined is coupled to the first space in the augmented reality space.

An extended or cross-reality system includes a computing device communicatively connected to a plurality of portable electronic devices via a network component; a repository accessible by the computing device and the plurality of portable electronic devices; and a dense map merging component. The extended or cross-reality system determines representations for the plurality of portions of the 3D environment based at least in part on a set of dense maps received from the plurality of portable devices, where the representations are determined for the plurality of portions of the 3D environment based at least in part on gesture data associated with the set of dense maps or surface information in the set of dense maps. A set of dense maps is grouped into a plurality of subgroups. The extended or cross-reality system stores the representation as at least a portion of a shared persistent dense map.