222 results about "Augmented reality systems" patented technology

An automation system controls an automated production process of an industrial plant. The industrial plant includes a plurality of equipment for performing the production process. The automation system includes an augmented reality system including augmented reality glasses, executable program logic and a database system including a task manager configured to store a list of processes associated with respective equipment, where each process includes one or more tasks to be performed with respect to the equipment. The executable program logic is configured to receive from the database system the list of processes for display to a user of the AR glasses, receive a selection of a displayed process assigned to the user, send the selected process to the database system, and receive from the database system additional data corresponding to the selected process.

This disclosure describes an interactive augmented reality system for improving surgeon's view and context awareness during laparoscopic and video assisted surgeries. Instead of purely relying on computer vision algorithms for image registration between pre-operation (or intra-operation) images / models and later intra-operation scope images, the system can implement an interactive mechanism where surgeons may provide supervised information in initial calibration phase of the augmented reality function, thus achieving high accuracy in image registration. Besides the initialization phase before operation starts, interaction between surgeon and the system can also happens during the surgery. Specifically, patient tissue might move or deform during surgery, caused by for example cutting. The augmented reality system can re-calibrate during surgery when image registration accuracy deteriorates, by seeking additional supervised labeling from surgeons. The augmented reality system can improve surgeon's view during surgery, by utilizing surgeon's guidance sporadically to achieve high image registration accuracy.

A virtual camera augmented reality (AR) system detects selection of a virtual camera added as an extension in an internet browser, generates a new canvas element for the virtual camera, and accesses video frames captured by a hardware camera coupled with a computing device. The virtual camera AR system detects selection of an AR option and applies the AR option to each video frame of at least a subset of the accessed video frames, at a predefined rate. The virtual camera AR system provides each video frame of at least the subset of video frames comprising the applied AR option to the new canvas element and causes display, of each video frame of at least the subset of video frames comprising the applied AR option in the new canvas element, on a user interface of the computing device.

An augmented reality display system comprises a passable world model data comprises a set of map points corresponding to one or more objects of the real world. The augmented reality system also comprises a processor to communicate with one or more individual augmented reality display systems to pass a portion of the passable world model data to the one or more individual augmented reality display systems, wherein the piece of the passable world model data is passed based at least in part on respective locations corresponding to the one or more individual augmented reality display systems.

An augmented reality system may include a waveguide having an edge surface extending between opposing light guide surfaces. The waveguide may direct light toward the edge surface. The waveguide may include a reduced reflectivity film, such as an absorber film or a photovoltaic film, disposed on the edge surface. To form a reduced reflectivity film, a curable material may be disposed on the dissolvable film. The curable material may be cured when disposed on the dissolvable film such that the cured material forms a reduced reflectivity structure on the dissolvable film. The dissolvable film may be dissolved such that the reduced reflectance structure remains intact as a reduced reflectance film, which may be adhered to the edge surface, for example, with a primer layer. The edge surface may include a nanostructure sized to be less than the half wavelength of the guided light, which nanostructure may reduce the reflectivity of the edge surface.

Systems and methods track a user's line-of-sight relative to objects through a transparent LCD display to provide information about the objects to be displayed on the display proximate the line-of-sight between the object being viewed and the user's head to create an augmented reality environment for the user. A method comprising: determining a position of an object; determining a position of a transparent display; determining a line-of-sight extending between a user's eye and the object through the transparent display; determining where the line-of-sight intersects the transparent display; and displaying object information corresponding to the object via the transparent display proximate where the line-of-sight intersects the transparent display.

A system and method for locating and guiding a peripheral nerve evaluation (PNE) lead. The system comprising an X-ray system to capture images of the patient. The images are processed by the system and are provided to a user to utilize an augmented reality (AR) system for inserting a needle for placing leads for the PNE procedure.

Systems and methods for interacting with virtual objects in a three-dimensional space using a wearable system are disclosed. The wearable system can be programmed to permit user interaction with interactable objects in a field of regard (FOR) of a user. The FOR includes a portion of the environment around the user that is capable of being perceived by the user via the AR system. The system can determine a group of interactable objects in the FOR of the user and determine a pose of the user. The system can update, based on a change in the pose or a field of view (FOV) of the user, a subgroup of the interactable objects that are located in the FOV of the user and receive a selection of a target interactable object from the subgroup of interactable objects. The system can initiate a selection event on the target interactable object.

A system and method for performing a structural heart repair or replacement procedure on a patient includes an augmented reality system, a tracked instrument, a first image acquisition system, and a computer system. The method includes a step of acquiring the first holographic image dataset from the patient. The computer system tracks, in a next step, the tracked instrument using the plurality of sensors to provide a tracked instrument dataset. The method also includes a step of registering the first holographic image dataset and the tracked instrument dataset with the patient. The augmented reality system then renders, in a next step, a first hologram based on the first holographic image dataset from the patient for viewing by a practitioner. The practitioner is thereby permitted to perform the procedure on the patient while viewing the patient and the first hologram with the augmented reality system.

The invention discloses a vehicle-mounted audio augmented reality system and method based on virtual-real fusion space anchoring, and relates to the technical field of intelligent cabin man-machine interaction, and the system comprises a controller, an augmented reality display device, a distributed loudspeaker array, and a sensor group. The controller receives the virtual image pixel coordinates, the eyeball position coordinates and the vehicle state semantic identifier, and stores a cabin three-dimensional digital model. The system converts a dynamic pixel coordinate into a three-dimensional virtual sound source coordinate by utilizing perspective inverse projection through vision-space mapping logic; the retrieval model anchors the semantic identifier to the physical component coordinates through the semantic-space mapping logic. The audio rendering module calculates a speaker drive gain based on the virtual sound source coordinates to synthesize a virtual sound source. By constructing a unified cabin three-dimensional digital model and a double-channel mapping mechanism, spatial alignment of an audio sound image, an AR visual track and a physical part of a vehicle body is realized, and audio-visual perception splitting is eliminated.

A system for generating an augmented reality effect using image data of a hand and a depth estimation model. A depth estimation model is trained using a pairing of the synthetic 2D image data with the depth set and the segmentation mask. An augmented reality system captures image data of a hand in a real-world scene and generates an augmented reality effect using the image data and a depth estimation model. Augmented reality effects are provided to a user during an augmented reality experience.

In an illustrative embodiment, systems and methods for an augmented reality (AR) system enable merchant-to-customer engagement through context-aware digital content placement. In an embodiment, one element of the system allows merchants to upload, customize, and place AR media objects, and another element allows customers to discover and interact with said content in real-world environments. In an embodiment, another element of the system dynamically ranks and serves AR content based on user behavior, demographics, time, and location. In alternative embodiments, an artificial intelligence engine dynamically generates personalized AR media objects based on real-time user preferences, history, and environmental context. In some embodiments, features include interactive merchant storefronts, digital promotions, real-time analytics, user rewards, breadcrumb-based AR trails, and dynamically personalized AR experiences to encourage repeated engagement.

An augmented reality system for controlling a controllable device located in an environment is disclosed. The augmented reality system includes: an augmented reality device including a display configured to present an indicator indicating a selection of a controllable device; an input interface configured to receive an input signal indicative of an environmental change in the environment; one or more sensors configured to provide one or more sensor signals indicative of a gaze direction of the user and / or an orientation direction of the augmented reality device; and a processor configured to determine, based on the one or more sensor signals from the one or more sensors, that a gaze direction of the user and / or an orientation direction of the augmented reality device has changed toward a target area within a predefined time period following an environmental change in the environment, where the target area is associated with a controllable device, a duration of a gaze of the user and / or an orientation of the augmented reality device towards the target area is determined, and if the duration exceeds a threshold duration, an indicator indicative of a selection of the controllable device is presented on the display, and if the duration does not exceed the threshold duration, the controllable device is presented on the display. If so, presentation of an indicator indicating a selection of the controllable device on the display is suppressed.

Systems and methods using augmented reality for dental evaluations. In some examples, the fit of dental appliances is evaluated. An error condition indicative of improper appliance fit on a patient's teeth may be identified from an image of the patient's teeth using an augmented reality system. A visual overlay identifying the error condition may be generated. The visual overlay may be output to a display of the augmented reality system. The visual overlay may be superimposed over a view of the patient's teeth and of the dental appliance placed on the patient's teeth.

Embodiments of the present disclosure generally relate to augmented reality systems. More specifically, embodiments described herein provide for augmented reality systems, methods of correcting an image projected into a waveguide, and related components. In one or more embodiments, an augmented reality system includes a light engine configured to emit an image and a waveguide including a substrate. The waveguide further includes an input coupler disposed over the substrate and configured to receive the image from the light engine. An output coupler is disposed over the substrate and configured to emit an outcoupled image. A controller is configured to generate corrective data based on one or more characteristics of the outcoupled image and adjust one or more components of the augmented reality system based on the corrective data.

The present disclosure includes methods (100, 200) for improving surgical accuracy and effectiveness in medical procedures by using an augmented reality system (300). The method (100, 200) involves: generating a preoperative plan in an augmented reality environment by visualizing a holographic representation of a patient's anatomy; recording program data according to the preoperative plan; using the recorded data to update the augmented reality system; and applying the updated system to establish a continuously improved feedback loop. According to the method, by integrating real-time data feedback into the operation process, the accuracy of the operation is improved, the effect of the patient is improved, the program is optimized, and continuous improvement of medical practice is promoted.

A method, system and product including obtaining a frame from a frame sensor of an Augmented Reality (AR) system comprising an AR device; executing a first skin on the frame or portion thereof to obtain a first overlay; executing a second skin on the frame or portion thereof to obtain a second overlay, wherein the first and second skins are configured to provide an augmented overlay to be used by the AR system in determining an AR view, wherein said executing the second skin is performed without providing the first overlay to the second skin; determining whether the second overlay can be utilized for providing the AR view; generating the AR view based on the first overlay and based on said determining whether the second overlay can be utilized for providing the AR view; and providing the AR view via the AR device.

Provided herein are systems, methods, and computer program products for determining partial impulsivity of a user of a virtual or augmented reality system. A virtual environment having a plurality of targets is provided. A visual stimulus is provided at a target. A first period of time from when the first visual stimulus is provided to when a user fixates on the visual stimulus, a first acceleration of the body part, a second period of time from when the first visual stimulus is provided to when the first acceleration occurs, a second acceleration (deceleration) of the body part, a third period of time from when the first acceleration occurs to when the velocity of the body part is zero is determined, a total fixation time on the visual stimulus, a total number of impulsive movements in a session, and an impulsivity score are determined.

A system for facilitating cardiopulmonary resuscitation (CPR) by a CPR performer on a patient includes a first sensor configured to be positioned at least partially between a hand of the CPR performer and a chest of the patient. The first sensor is configured to measure a force exerted on the chest of the patient by the hand of the CPR performer. The system also includes a computing system configured to: (1) receive data from the first sensor; (2) compare the data to stored data in a library that corresponds to the data received from the first sensor; and (3) generate one or more outputs in response to the comparison.

An augmented reality system includes an augmented reality (AR) device and a server computing device. The AR device transmits sensor data to the server computing device. The server computing device identifies a person and / or object in the sensor data. The server computing device executes a search over social network data to identify a portion of the social network data that relates to the identified person and / or object. The server computing device outputs augmentation data to the AR device that includes the identified portion of the social network data, and is configured to cause the AR device to display an augmentation that includes the portion of the social network data.

The present invention relates to a method and device for alerting drivers to oncoming traffic in the same lane. The method and device use an augmented reality system to alert the driver of a vehicle that another vehicle is approaching from the opposite direction in the same lane. Images are displayed by this augmented reality system. These images include graphic objects that represent the imminence of a collision between the two vehicles and the danger of the road situation. (See Figure 8 for abbreviations.)

An augmented reality system can include a headset frame wearable by a user. A window disposed in or on the headset frame can include adjacent first and second window regions. When the headset frame is worn by the user, a field of view of an eye of the user can extend through at least part of the first window region. A camera can be disposed in or on the headset frame. When the headset frame is worn by the user, the camera can have a field of view that is directed away from the user and through the second window region. A controllably dimmable filter can be located adjacent to the window in the field of view of the eye and the field of view of the camera. The controllably dimmable filter can have a controllable opacity that determines a visibility of the camera through the second window region.

An augmented reality system can include a waveguide having an edge surface that extends between opposing light-guiding surfaces. The waveguide can guide light toward the edge surface. The waveguide can include a reflectivity-reducing film, such as an absorptive film or a photovoltaic film, disposed on the edge surface. To form the reflectivity-reducing film, curable material can be disposed onto a dissolvable film. The curable material can be cured while disposed on the dissolvable film such that the cured material forms a reflectivity-reducing structure on the dissolvable film. The dissolvable film can be dissolved such that the reflectivity-reducing structure remains intact as a reflectivity-reducing film that can be adhered to the edge surface, such as with a primer layer. The edge surface can include nanostructures, sized smaller than half a wavelength of the guided light, that can reduce a reflectivity of the edge surface.

Augmented reality systems provide graphics over views from a mobile device for both in-venue and remote viewing of a sporting or other event. A server system can provide a transformation between the coordinate system of a mobile device (mobile phone, tablet computer, head mounted display) and a real world coordinate system. Requested graphics for the event are displayed over a view of an event. In a tabletop presentation, video of the event can be displayed with augmented reality graphics overlays at a remote location.

Systems and methods for social networking using augmented reality are disclosed. A network can receive requests from a plurality of client devices to connect to a server. The server can be associated with an object, such as a landmark or a product. The server can serialize an AR world including the object and content associated with the object and send data indicative of the AR world to the plurality of client devices. The server can receive, from a first client device of the plurality of client devices via the network, an indication of user input associated with the AR world. The indication of user input can include textual data, digital image data, or user feedback. The indication of user input can be broadcast to the plurality of client devices.

The present invention discloses a mixed reality system comprising a display and an optical system that projects the image into a viewer's eye where the display is convex or a concave optical component. The display may further be comprising of smaller displays bunched together to create optical functions. In addition, the display pixels of further comprise of sub-pixels.

The augmented reality system includes a projector and projection optics coupled to the projector. The augmented reality system also includes an eyepiece waveguide including a hybrid diffractive structure including one or more first nanofeatures having a first material with a first refractive index and one or more second nanofeatures having a second material with a second refractive index.

One example of the present application discloses an augmented reality (AR) method and system which may connect a remote expert with a front line worker. A first video stream from a 2D camera at a first location where the remote expert is located is processed in real time to determine 3D coordinates of a hand of the remote expert. An AR model is mapped to the 3D coordinates of the hand of the remote expert and rendering in real time as a first AR image in a AR headset at a second location at which the front line worker is located. Further examples relate to an augmented lighting system and an augmented optics system.

In one embodiment, a method includes rendering, for one or more displays of a XR display device, a first output image of an XR object within an XR environment in a field of view (FOV) of a first user, where the XR object has a first form, detecting a change in a context of the user with respect to the XR object, determining, based on the detected change in the context of the user, whether to invoke an attention system with respect to the XR object, and rendering, for the one or more displays of the XR display device, a second output image of the XR object responsive to invoking the attention system, where the XR object is morphed to have a second form indicating a first attention state, where the first attention state indicates a status of the XR object to interact with one or more first voice commands.