333 results about "Augmented reality systems" patented technology

The invention relates to the technical field of augmented reality application, and discloses an augmented reality system and method for immersive experience. The method comprises the following steps: collecting dynamic spatial data of a physical environment in real time, and synchronously obtaining an interactive behavior data stream of a user terminal; performing environment semantic modeling on the dynamic spatial data to generate multi-level feature description of a physical environment, and determining a rendering priority sequence of virtual-real superposition content according to the multi-level feature description; extracting behavior pattern characteristics in historical interaction data of the user, performing immersion correlation analysis on the behavior pattern characteristics to obtain environment cognitive preference parameters of the current user, and generating a virtual-real interaction event sequence in combination with the environment cognitive preference parameters and the interaction behavior data flow; and dynamically compensating the virtual-real synchronization error in the immersive experience process according to the rendering priority sequence and the virtual-real interaction event sequence. According to the method, the accuracy and naturalness of virtual-real fusion can be improved, the personalized requirements of the user are met, and the overall immersive experience effect is enhanced.

An automation system controls an automated production process of an industrial plant. The industrial plant includes a plurality of equipment for performing the production process. The automation system includes an augmented reality system including augmented reality glasses, executable program logic and a database system including a task manager configured to store a list of processes associated with respective equipment, where each process includes one or more tasks to be performed with respect to the equipment. The executable program logic is configured to receive from the database system the list of processes for display to a user of the AR glasses, receive a selection of a displayed process assigned to the user, send the selected process to the database system, and receive from the database system additional data corresponding to the selected process.

A system can operate to receive an indication of a point in a space of an environment, wherein an augmented reality system is located in the space of the environment at a coordinate in a first coordinate system. The system can operate to query a digital twin for a coordinate of the point in a second coordinate system, the digital twin including a contextual description of the environment. The system can operate to translate the coordinate of the augmented reality system in the first coordinate system to a coordinate of the augmented reality system in the second coordinate system based on the coordinate of the point in the second coordinate system and the coordinate of the augmented reality system in the first coordinate system. The system can operate to generate data to cause the augmented reality system to display information based at least in part on the translation.

Devices, networks, systems, methods, and processes for an Augmented Reality (AR) based device management. An AR device may receive an image depicting one or more objects, from a camera. The AR device may determine position data, including a position and an orientation, of the AR device, and transmit the position data and the image to an application server. The application server may identify the one or more objects in the image and obtain status data associated with the one or more objects. The application server may generate overlay data based on the status data of the objects. The application server may transmit the overlay data to the AR device. An augmented image generated based on the overlay data may be displayed on the AR device to present the object status data. Such augmented images integrate information with real-world objects, providing instant access to object status data and enhancing user understanding.

This disclosure describes an interactive augmented reality system for improving surgeon's view and context awareness during laparoscopic and video assisted surgeries. Instead of purely relying on computer vision algorithms for image registration between pre-operation (or intra-operation) images / models and later intra-operation scope images, the system can implement an interactive mechanism where surgeons may provide supervised information in initial calibration phase of the augmented reality function, thus achieving high accuracy in image registration. Besides the initialization phase before operation starts, interaction between surgeon and the system can also happens during the surgery. Specifically, patient tissue might move or deform during surgery, caused by for example cutting. The augmented reality system can re-calibrate during surgery when image registration accuracy deteriorates, by seeking additional supervised labeling from surgeons. The augmented reality system can improve surgeon's view during surgery, by utilizing surgeon's guidance sporadically to achieve high image registration accuracy.

A head mounted display for use in a virtual reality, mixed reality or augmented reality system comprises a display device configured to display an image, a tracking system configured to determine a gaze direction of a user's eye and an optical system configured to project the image into the user's eye, the optical system comprising an adjustable optical device; and processing circuitry configured to control the adjustable optical device based on the gaze direction of the user's eye such that a position of the image projected into the user's eye is unchanged, when the gaze direction of the user's eye changes.

A virtual camera augmented reality (AR) system detects selection of a virtual camera added as an extension in an internet browser, generates a new canvas element for the virtual camera, and accesses video frames captured by a hardware camera coupled with a computing device. The virtual camera AR system detects selection of an AR option and applies the AR option to each video frame of at least a subset of the accessed video frames, at a predefined rate. The virtual camera AR system provides each video frame of at least the subset of video frames comprising the applied AR option to the new canvas element and causes display, of each video frame of at least the subset of video frames comprising the applied AR option in the new canvas element, on a user interface of the computing device.

An augmented reality display system comprises a passable world model data comprises a set of map points corresponding to one or more objects of the real world. The augmented reality system also comprises a processor to communicate with one or more individual augmented reality display systems to pass a portion of the passable world model data to the one or more individual augmented reality display systems, wherein the piece of the passable world model data is passed based at least in part on respective locations corresponding to the one or more individual augmented reality display systems.

An eyewear device having an image processor operable in a camera pipeline for computer vision (CV) and in augmented reality (AR) systems. The image processor is configured to selectively control a plurality of cameras to provide images having a first resolution in the high power AR mode, and to provide the images having a second resolution in the low power CV mode. The first resolution is higher than the second resolution, and the plurality of cameras consume less power in the CV mode than the AR mode. The image processor controls the camera pipeline to process the first resolution high IQ images from the plurality of cameras to operate in the AR mode, and controls the camera pipeline to process the second resolution lower IQ images from the plurality of cameras to operate in the CV mode. Substantial power is saved by reducing the resolution of the images using downscaling in the cameras themselves in the CV mode.

An augmented reality (AR) system for an aircraft obtains an aircraft pose, determines a passenger pose relative to a window, and computes a field-of-view of the passenger. Operations select a set of passenger-relevant points of information (POIs) from a database that satisfy a relevance rule to the real-world objects viewable by the passenger through the window. For each of the passenger-relevant POIs in the set, operations determine an AR indicia to be displayed, and an AR indicia pose where the AR indicia is to be rendered on a display device. Operations generate a display command based on the AR indicia and the AR indicia pose for the passenger-relevant POIs in the set. Operations provide the display command to the display device to display renderings of the AR indicias with the AR indicia poses to augment the real-world objects viewed by the passenger through the window.

An augmented reality system may include a waveguide having an edge surface extending between opposing light guide surfaces. The waveguide may direct light toward the edge surface. The waveguide may include a reduced reflectivity film, such as an absorber film or a photovoltaic film, disposed on the edge surface. To form a reduced reflectivity film, a curable material may be disposed on the dissolvable film. The curable material may be cured when disposed on the dissolvable film such that the cured material forms a reduced reflectivity structure on the dissolvable film. The dissolvable film may be dissolved such that the reduced reflectance structure remains intact as a reduced reflectance film, which may be adhered to the edge surface, for example, with a primer layer. The edge surface may include a nanostructure sized to be less than the half wavelength of the guided light, which nanostructure may reduce the reflectivity of the edge surface.

Systems and methods track a user's line-of-sight relative to objects through a transparent LCD display to provide information about the objects to be displayed on the display proximate the line-of-sight between the object being viewed and the user's head to create an augmented reality environment for the user. A method comprising: determining a position of an object; determining a position of a transparent display; determining a line-of-sight extending between a user's eye and the object through the transparent display; determining where the line-of-sight intersects the transparent display; and displaying object information corresponding to the object via the transparent display proximate where the line-of-sight intersects the transparent display.

A system and method for locating and guiding a peripheral nerve evaluation (PNE) lead. The system comprising an X-ray system to capture images of the patient. The images are processed by the system and are provided to a user to utilize an augmented reality (AR) system for inserting a needle for placing leads for the PNE procedure.

Systems and methods for interacting with virtual objects in a three-dimensional space using a wearable system are disclosed. The wearable system can be programmed to permit user interaction with interactable objects in a field of regard (FOR) of a user. The FOR includes a portion of the environment around the user that is capable of being perceived by the user via the AR system. The system can determine a group of interactable objects in the FOR of the user and determine a pose of the user. The system can update, based on a change in the pose or a field of view (FOV) of the user, a subgroup of the interactable objects that are located in the FOV of the user and receive a selection of a target interactable object from the subgroup of interactable objects. The system can initiate a selection event on the target interactable object.

Devices, networks, systems, methods, and processes for an Augmented Reality (AR) based device management. An AR device of the present disclosure can receive an image from a camera. The image may depict one or more objects. The AR device can determine device position data, including a position and an orientation, of the AR device. The AR device can identify the one or more objects in the image based on the device position data. The AR device may identify one or more controllers associated with the objects. The AR device can retrieve control data associated with the objects from the one or more controllers. The AR device may superimpose the control data on the image to generate an augmented image. The AR device can display the augmented image to a user. The AR device may further generate one or more control signals associated with the one or more objects.

A system and method for performing a structural heart repair or replacement procedure on a patient includes an augmented reality system, a tracked instrument, a first image acquisition system, and a computer system. The method includes a step of acquiring the first holographic image dataset from the patient. The computer system tracks, in a next step, the tracked instrument using the plurality of sensors to provide a tracked instrument dataset. The method also includes a step of registering the first holographic image dataset and the tracked instrument dataset with the patient. The augmented reality system then renders, in a next step, a first hologram based on the first holographic image dataset from the patient for viewing by a practitioner. The practitioner is thereby permitted to perform the procedure on the patient while viewing the patient and the first hologram with the augmented reality system.

The invention discloses a vehicle-mounted audio augmented reality system and method based on virtual-real fusion space anchoring, and relates to the technical field of intelligent cabin man-machine interaction, and the system comprises a controller, an augmented reality display device, a distributed loudspeaker array, and a sensor group. The controller receives the virtual image pixel coordinates, the eyeball position coordinates and the vehicle state semantic identifier, and stores a cabin three-dimensional digital model. The system converts a dynamic pixel coordinate into a three-dimensional virtual sound source coordinate by utilizing perspective inverse projection through vision-space mapping logic; the retrieval model anchors the semantic identifier to the physical component coordinates through the semantic-space mapping logic. The audio rendering module calculates a speaker drive gain based on the virtual sound source coordinates to synthesize a virtual sound source. By constructing a unified cabin three-dimensional digital model and a double-channel mapping mechanism, spatial alignment of an audio sound image, an AR visual track and a physical part of a vehicle body is realized, and audio-visual perception splitting is eliminated.

An augmented reality system is provided, which includes: a management platform; a first user terminal communicating with the management platform, which includes: a positioning module for obtaining positioning information; an imaging module for capturing an environmental image from a physical environment; and a display module for displaying a virtual placement according to the positioning information and physical address information corresponding to the environmental image, wherein the virtual placement includes a first virtual object; and a setting terminal communicating with the management platform, which includes: a input module for inputting the physical address information, an identification image of an identification object, and the first virtual object; wherein the management platform sets the virtual placement according to the physical address information, the identification image, and the first virtual object.

The invention discloses an unmanned aerial vehicle virtual-real fusion training system and method based on augmented reality, and belongs to the technical field of unmanned aerial vehicle simulation training. An augmented reality-based unmanned aerial vehicle virtual-real fusion training system comprises an unmanned aerial vehicle simulator and an augmented reality system based on a first visual angle; the unmanned aerial vehicle simulator comprises an unmanned aerial vehicle model, a flight control module, a remote controller and a sensor module. The augmented reality system based on the first visual angle comprises a head-mounted AR device, an image recognition and tracking module and a virtual information generation and fusion module. The unmanned aerial vehicle virtual-real fusion training method adopts the unmanned aerial vehicle virtual-real fusion training system for training. According to the invention, virtual training information and a real scene are deeply fused through the AR technology, and students can obtain visual and spatial perception experience close to real flight.

Systems and methods for providing context-based content display are provided. The system may include a device having a display that provides content in a viewing area of the display. The device may include a camera device configured to track a scene of a real-world environment acquired in a field of view of the camera device. The device may determine a region of interest in the scene. The device may perform object recognition on a scene tracked by the camera device. The device may determine an environmental interaction based on the object recognition and the region of interest. The device may adaptively change the location of the content provided by the display based on the environmental interaction.

An augment reality (AR) system captures an image of a physical environment. The AR system identifies an object in the captured image to serve as an anchor point. The AR system calculates a distance between the identified object and an AR display device that comprises left and right displays. The AR system identifying a virtual object associated with the anchor point. The AR system then generates for simultaneous display: (a) a first separate image of the virtual object on the left display of the AR device, and (b) a second separate image of the virtual object on the right display of the AR device, such that apparent distance of the virtual object of the composite image of the first separate image and the second separate image is set to the calculated distance between the identified object and the AR display device.

An augmented reality system includes a light source to generate a virtual light beam, the virtual light beam carrying information for a virtual object. The system also includes a light guiding optical element, the light guiding optical element allowing a first portion of a first real-world light beam to pass therethrough, where the virtual light beam enters the light guiding optical element, propagates through the light guiding optical element by substantially total internal reflection (TIR), and exits the light guiding optical element. The system further includes a lens disposed adjacent and exterior to a surface of the light guiding optical element, the lens comprising a light modulating mechanism to absorb a second portion of the real-world light beam and to allow the first portion of the real-world light to pass through the lens.

Systems and methods are described herein for controlling network devices in an augmented reality environment. A user may point a second network device at a first network device to determine a network activity the first network device. The second network device may display a user control interface to enable the user to control the network activity of the first network device (e.g., a pinch gesture control). In response to receiving the user input, the second network device causes the modification of the network activity based on the user input.

An augmented reality system to generate and cause display of a presentation of a space at a first client device, receive one or more selections of points within the presentation of the space at the first client device, and render graphical elements at the one or more points within the presentation of the space at the first client device. The augmented reality system is further configured to receive a display request to display the space at a second client device, and in response, may render a second presentation of the space at the second client device, wherein the second presentation of the space includes the graphical elements at the one or more points.

Systems and methods are described herein for controlling network devices in an augmented reality environment. A user may point a second network device at a first network device to determine a network activity the first network device. The second network device may display a user control interface to enable the user to control the network activity of the first network device (e.g., a pinch gesture control). In response to receiving the user input, the second network device causes the modification of the network activity based on the user input.

The invention provides a color characterization method for an optical perspective augmented reality system, and the method comprises the steps: carrying out the full-view-field measurement of different environment illumination conditions and OST AR virtual stimulation colors, building a mathematic relation between a virtual display color which should be outputted by AR display, a human eye perception color, and ambient light, and enabling the mathematic relation to describe the color characterization of the virtual display color under different illumination conditions. According to AR display, how to adjust own color output is used for compensating the influence of ambient light on final perception colors, so that the accuracy and consistency of color reproduction are improved; that is to say, the nonlinear influence of the ambient light on the AR virtual display content is corrected through the nonlinear mixed compensation model of the ambient light measurement and the OST AR virtual stimulation color, and the human eye perception color deviation of the OST AR equipment in the complex illumination environment is corrected. The color consistency of OST AR display in an illumination environment and the color rendition capability of an OST AR system under different illumination conditions are improved, and a more natural virtual-real fusion effect is achieved.

An augmented reality system includes a projector and projection optics coupled to the projector. The augmented reality system also includes an eyepiece waveguide including hybrid diffractive structure including: one or more first nanofeatures having first material with a first index of refraction; and one or more second nanofeatures having a second material with a second index of refraction.

A system for generating an augmented reality effect using image data of a hand and a depth estimation model. A depth estimation model is trained using a pairing of the synthetic 2D image data with the depth set and the segmentation mask. An augmented reality system captures image data of a hand in a real-world scene and generates an augmented reality effect using the image data and a depth estimation model. Augmented reality effects are provided to a user during an augmented reality experience.

In an illustrative embodiment, systems and methods for an augmented reality (AR) system enable merchant-to-customer engagement through context-aware digital content placement. In an embodiment, one element of the system allows merchants to upload, customize, and place AR media objects, and another element allows customers to discover and interact with said content in real-world environments. In an embodiment, another element of the system dynamically ranks and serves AR content based on user behavior, demographics, time, and location. In alternative embodiments, an artificial intelligence engine dynamically generates personalized AR media objects based on real-time user preferences, history, and environmental context. In some embodiments, features include interactive merchant storefronts, digital promotions, real-time analytics, user rewards, breadcrumb-based AR trails, and dynamically personalized AR experiences to encourage repeated engagement.