1645 results about "Mixed reality" patented technology

The invention relates to the technical field of artificial intelligence assisted medical rehabilitation, in particular to a peripheral nerve injury personalized rehabilitation system and method based on a multi-modal large model, and the method comprises the steps: a feature fusion module employs space-time attention to fuse multi-modal time series data, and constructs an evaluation map; the personalized generation module is combined with historical data and reinforcement learning to generate a scheme containing virtual scene parameters; the interaction feedback module collects data through mixed reality and calculates action deviation; and the adaptive adjustment module adopts a meta-learning optimization model and a distributed iterative output scheme. According to the method, the cross-modal association precision of the motion features and the mechanical parameters is improved, dynamic matching of the training scene and the motion ability of the user is achieved, the virtual environment and the mechanical feedback threshold are optimized by dynamically adjusting the rehabilitation scheme parameters, the scheme optimization period is shortened based on an online iterative optimization mechanism, and the training efficiency is improved. The core defects of personalized adaptation lagging and low utilization efficiency of multi-modal data are overcome.

The invention discloses an electrical equipment operation state monitoring method, system and terminal, full life cycle health management of equipment is realized through multi-dimensional perception and intelligent analysis, a composite sensor network can be constructed from the level of the method, and high-frequency current, ultrahigh frequency, fiber grating temperature vibration, multi-parameter gas and acoustic sensors are integrated. Electromagnetic characteristics, mechanical states, environmental parameters and voiceprint characteristics are covered; the adaptive signal processing technology performs classification and noise reduction on multi-source data, and the three-dimensional digital twin model realizes time-space fusion of a temperature field, a vibration field, an electric field and a sound field; a lightweight space-time convolutional network is deployed to fuse a time domain waveform, a spectrogram and spatial distribution characteristics for diagnosis, and a hidden semi-Markov model and a particle filter algorithm are combined to dynamically predict the residual service life of equipment. The system architecture comprises a self-organizing sensor network with edge computing capability, a time-sensitive industrial communication network and a containerized analysis engine, and supports mixed reality visual interaction.

The invention relates to the technical field of mixed reality, in particular to a mixed reality multi-user interaction large-space positioning, aligning and collaborating system and method, and aims to solve the problems that in the prior art, the user interaction experience is improved only by collecting multi-aspect data for processing, efficient collaboration work of multiple users in a large space cannot be achieved, and the user experience is poor. Information sharing, task distribution and real-time interaction among users cannot be realized; information sharing, task distribution and real-time interaction among users are realized through the multi-user cooperation module, cross-device operation consistency is ensured through a multi-device sharing coordinate system, configuration is simplified, cooperation efficiency is improved, real-time data transmission is realized through multi-device network synchronization, delay is reduced, cooperation fluency is enhanced, and user experience is improved. The interaction reality sense is enhanced through physical interaction logic synchronization, the working process is optimized, and the cooperation experience is improved.

The invention relates to the technical field of data processing, and discloses a comprehensive management method and system for a smart park, and a storage medium. The method comprises the steps that environment, energy consumption and people flow data are collected through the Internet of Things, and a basic data set is obtained through edge node processing and semantic alignment; constructing a digital twin system by using a spatio-temporal data cube and a graphical mode; constructing a land digital identity system by adopting space-time coding and block chain technologies; performing anomaly identification and risk analysis on the activity data to obtain supervision early warning data; performing multi-dimensional analysis on the enterprise operation data to form an enterprise portrait; and presenting resource distribution through mixed reality, and analyzing cross-department data to obtain operation efficiency data. According to the invention, holographic perception, intelligent supervision and cooperative service of park elements are realized.

Methods and systems are described for extended and mixed reality experience adaptation, processing, and handover from one computing or processing entity to another. In response to a change in a condition impacting resource usage, delivery of content is transferred from one edge node to another. One or more changes in a viewing client, a first edge node, a second edge node, a communication network, and content signal a handover. Artificial intelligence systems, including neural networks, and models are trained and developed for improving the adaptation, processing, and handover. Related apparatuses, devices, techniques, and articles are also described.

In one embodiment, a method includes receiving, by a mixed reality (MR) display device, an audio input from a first user of the MR display device, where the MR display device is associated with an MR environment including several MR objects, processing, using a natural language understanding (NLU) model, the audio input to identify one or more intents and one or more slots associated with the audio input, identifying a first MR object from several MR objects that is in an active listening state, where the first MR object is associated with a first set of intents and a first set of slots, determining that either the first set of intents or the first set of slots do not include the one or more identified intents or the one or more identified slots associated with the audio input, and generating, using a large language model (LLM), an out-of-domain (OOD) response.

The invention relates to the field of man-machine interaction training, virtual reality and intelligent evaluation, and particularly discloses a psychological exercise ability training method and system based on mixed reality and behavior analysis. The method comprises the steps that a multi-scene psychological training environment is constructed and presented in mixed reality equipment, action behavior data, eye movement path data and operation response time data of a pilot in the training process are collected, multi-modal original behavior characteristics are constructed, and then a psychological exercise ability label set is constructed; and realizing capability-task modeling through matching of a capability label and a task structure, generating a personalized training path and a capability difference atlas based on difference analysis with a preset capability standard, adjusting task parameter configuration data, dynamically updating a training scene, generating a capability evolution trajectory, and outputting a training analysis result. In this way, accurate ability modeling, task dynamic adaptation and visual tracking of the training effect can be achieved, and the individuation and intelligence level of psychological exercise ability training is remarkably improved.

To provide an information processing system, a control method for the information processing system, and a program that allow a user to distinguish a sound from a specific sound source while leaving the sounds from each sound source as they are, even when a plurality of sound sources are present in a direction of interest that is of interest to the user.SOLUTION: An information processing system 1000 comprises: display means (a display unit 107) capable of displaying an image of a mixed reality space; playback means (an audio output unit 105) capable of playing back sounds from a plurality of sound sources to a user; vibration applying means (a vibration unit 117) for applying vibration to the user; control means (a control unit 111) for controlling displaying a virtual object that can be linked to each sound source on the display means; and selection means for selecting the virtual object (an object selection instruction unit 116). The vibration applying means applies vibration to the user in accordance with the intensity of the sound from the sound source associated with the virtual object if the virtual object is selected.SELECTED DRAWING: Figure 2

The invention relates to the technical field of image processing, and discloses a virtual operation analogue simulation method and system based on a mixed reality technology. The method comprises the steps that three-dimensional space information is collected, feature extraction and registration are conducted on a real operation environment and video data, and mixed reality mapping and monitoring data are generated; performing medical image registration processing based on mixed reality mapping to obtain virtual scene data; processing instrument sensing data according to the mapping data and the monitoring parameters to obtain interaction data; performing surgical process simulation according to the virtual scene and the interaction data; performing operation trajectory analysis and identification based on the simulation data, and generating evaluation and early warning information; and analyzing the evaluation and early warning information, and outputting an analysis report. Through multi-source data fusion and a real-time evaluation feedback mechanism, the authenticity and accuracy of operation simulation training are effectively improved.

The invention relates to the technical field of automobile education virtual practical training, in particular to a virtual practical training method for automobile vocational education. According to the technical scheme, the method comprises the following steps: initializing a multi-modal interaction system, specifically comprising the following steps: starting a virtual reality module, and creating an automobile practical training scene, including whole automobile internal structure display, engine operation simulation and automobile driving environment simulation; starting an augmented reality module, and preparing to superpose virtual information on a real automobile model; and starting the mixed reality module. Through the steps of multi-modal interaction system initialization, trainee practical training operation, intelligent cooperative tutoring network analysis, ability evaluation and course adjustment, dynamic updating and continuous learning and virtual competition module setting, a vivid practical training scene, personalized tutoring and learning paths are provided for trainees, learning enthusiasm and competition awareness are stimulated, and the training efficiency is improved. The ability of trainees is accurately evaluated, the teaching content is ensured to closely follow the industry development, and the effect of automobile vocational education is significantly improved.

The invention relates to a nurse handover management system and method, and relates to the technical field of data processing, and the system comprises a real-time monitoring module which is used for constructing an individualized baseline database of a patient; the data integration module is used for outputting a multi-dimensional structured data set; the intelligent analysis engine is used for identifying clinical state change information of the patient and generating a corresponding risk early warning score; the decision support module is used for generating targeted nursing intervention suggestions; the interactive shift exchange platform is used for generating a structured shift exchange abstract based on the risk early warning score, the clinical change information and the nursing intervention suggestion, and dynamically displaying the structured shift exchange abstract through a mixed reality technology; and the closed-loop optimization module is used for receiving shift-change feedback information and subsequent execution data of the shift-change party, evaluating the completeness of shift-change information, updating a nursing plan, and optimizing and adjusting analysis and recommendation parameters of the system. The system can improve the accuracy and efficiency of nurse handover management.

In one embodiment, a method includes capturing images of a first user wearing a VR display device in a real-world environment. The method includes receiving a VR rendering of a VR environment. The VR rendering is from the perspective of the mobile computing device with respect to the VR display device. The method includes generating a first MR rendering of the first user in the VR environment., The first MR rendering of the first user is based on a compositing of the images of the first user and the VR rendering. The method includes receiving an indication of a user interaction with one or more elements of the VR environment in the first MR rendering. The method includes generating, in real-time responsive to the indication of the user interaction with the one or more elements, a second MR rendering of the first user in the VR environment. The one or more elements are modified according to the interaction.

An augmented reality (AR) or mixed reality (MR) eyewear display utilizes an antenna arrangement for eyewear that optimizes the performance of the antenna without sacrificing the functionality or modularity of the eyewear. In one implementation, the antenna arrangement includes a first conductor that functions as a ground for the antenna, a second conductor configured to help retain a lens, and an antenna feed configured to excite the second conductor, causing the second conductor to act as an antenna. The first and / or second conductor additionally acts as a lens retention feature by interfacing with a groove in a lens and the use of a locking clip that prevents the lens from becoming dislodged from the second conductor, thus enabling quick and easy installation or removal of the lens.

Methods and system for atrial fibrillation ablations and other cardiac procedures utilizes a cardiac mapping system for navigation and guidance, and includes an additional virtual reality (VR) system and an augmented reality (AR) or mixed reality (MR) system. The Cardiac mapping system utilizes medical images and patient's electrical signals. The virtual reality (VR) system is utilized pre-procedure for visualization and pre-procedure planning. The augmented reality (AR) or mixed reality (MR) system is utilized pre-procedure or intra-procedure. Methods and system are disclosed utilizing pre-built virtual device models which interact with 3D volume rendered structures from patient's CT or MRI of the region of interest (ROI) for the different type of cardiac procedure. With virtual reality (VR) the operator utilizes hand-held sensors. With augmented reality (AR) or mixed reality (MR) a combination of hand gestures and an X-box controller is used to manipulate virtual devices, anatomical structures in the Hologram.

Particular embodiments are directed to passthrough image generation for a mixed-reality experience. A device may determine, using an eye-tracking system of a head-mounted device, eye-tracking data associated with a user of the head-mounted device. The device may determine, based on the eye-tracking data, a desired scene depth for the user. The device may instruct a first autofocus camera of the head-mounted device to adjust a first focus distance based on the desired scene depth and capture a first image of a real-world environment of the user. The device may generate a first passthrough image based on the first image. The device may display the first passthrough image to a first eye of the user via a first display of the head-mounted device.

This disclosure and exemplary embodiments described herein provide a Parallel Content Authoring Method and Tool for Procedural Guidance, and a Remote Expert Method and System Utilizing Quantitative Quality Assurance in Mixed Reality. The implementation described herein is related to the generation of content / instruction set that can be viewed in different modalities, including but not limited to mixed reality, VR, audio text, however it is to be understood that the scope of this disclosure is not limited to such application.

The disclosed method encompasses performing local dimming of a dimmed area within a display area of mixed reality glasses. The dimmed area comprises a part of the display area on which a virtual object is displayed. While local dimming increases perceptibility of the virtual object, it might impair usability since it blocks real world objects from view. This is avoided by reducing a dimming intensity of the local dimming in at least a part of the dimmed area.

This invention is an optical structure with a plurality of movable reflective elements. These reflective elements have first configurations which allow transmission of environmental light. These reflective elements have second configurations which block (or reduce) transmission of environmental light and which reflect light from a virtual-object display toward a person's eye. This enables display of a relatively-opaque virtual object in an area of a person's field of view, while providing the person with a relatively-clear view of their environment in the rest of their field of view.

The invention discloses an intelligent home decoration system and method based on mixed reality, and the system comprises a data collection module which comprises a space scanning module, a 3D model generation module and a space feature recognition module; the intelligent assistant module comprises an AI drive design suggestion module based on deep learning, an automatic layout optimization module, a personalized recommendation engine and a design rule check module; the multi-party cooperation module comprises a cross-device synchronization module, a real-time multi-user interaction module, a remote cooperation support module and a role authority management module; and the visual presentation module comprises a mixed reality rendering module, a real-time light and shadow simulation module, a material reality presentation module and a virtual image presentation module. According to the method, the design efficiency is greatly improved, the design period is shortened, design errors and rework are reduced, the project cost is reduced, the design quality is improved, the whole-course visualization and traceability of the design process are realized, and the working mode of traditional home decoration design is changed.

A surgical planning system for use in surgical procedures to repair an anatomy of interest includes a preplanning system to generate a virtual surgical plan and a mixed reality system that includes a visualization device wearable by a user to view the virtual surgical plan projected in a real environment. The virtual surgical plan includes a 3D virtual model of the anatomy of interest. When wearing the visualization device, the user can align the 3D virtual model with the real anatomy of interest, thereby achieving a registration between details of the virtual surgical plan and the real anatomy of interest. The registration enables a surgeon to implement the virtual surgical plan on the real anatomy of interest without the use of tracking markers.

The invention relates to the field of training simulation, and discloses a public security AI-driven data generation type investigation teaching practical training system, which comprises the following modules: a case dynamic generation module, which is used for receiving original data of a public security case database, generating a logically closed virtual case through tensor decomposition, and sending the virtual case to the public security case database; transmitting the generated case data to a multi-modal virtual-real fusion training module; and the multi-modal virtual-real fusion training module is used for receiving the virtual case data, realizing interactive training of multi-modal clues through a mixed reality technology, dynamically injecting interference clues, and transmitting student operation data to the causal reinforcement learning evaluation module. Through the collaborative technical scheme of tensor decomposition and nonlinear constraint optimization, the integrity and space-time consistency generation of the virtual case evidence chain is realized, and compared with a traditional case generation method based on a fixed template, the problem of contradiction between material evidence association fracture and behavior logic is effectively solved.

The invention belongs to the technical field of wind power generation, and discloses a wind power plant digital twin modeling system and a use method, the system comprises an equipment module, the equipment module comprises a biological bionic adaptive fan blade system, a quantum sensing network, a laser radar system, a high-resolution multi-modal imaging system and an intelligent autonomous flying robot group system; the network module comprises a 5G private network, a quantum communication network and an edge computing network; the data module comprises a distributed database system, a big data processing platform and an AI training platform; the AI module comprises a wind regime prediction model, a power prediction model, an equipment health state monitoring model and a self-adaptive control algorithm model; the application module comprises a digital twinborn visualization platform, a holographic mixed reality operation and maintenance system and an intelligent scheduling system. Modeling and intelligent management of the wind power plant can be realized, the power generation efficiency can be improved, the operation and maintenance cost can be reduced, and the service life of equipment can be prolonged.

The invention provides a low-altitude typical scene-oriented aircraft cockpit simulation method, system and device and a storage medium, and the method comprises the steps: constructing a dynamic extensible low-altitude scene library through fusing mixed reality (MR) and a high-precision digital twin model; in combination with a man-machine in-the-loop multimode interaction technology (eye movement-gesture-voice cooperation), the operation naturalness is improved; and an intelligent optimization algorithm driven by reinforcement learning is introduced, adaptive adjustment of flight control parameters is realized, and finally a'perception-decision-evaluation 'closed-loop training system is formed. According to the invention, digital twinning, mixed reality (MR), multi-modal sensing and adaptive control technologies are fused, pilot low-altitude flight skill training, man-machine work efficiency evaluation and intelligent aircraft control system verification are carried out, and the method can be expanded to UAM operator training and unmanned aircraft simulation testing. The technical barrier of low-altitude flight skill training at present is solved, and the technical blank of low-altitude dynamic scene simulation is filled up.

The invention belongs to the technical field of building construction safety management, and provides a construction safety early warning method and system based on mixed reality, and the method comprises the steps: converting the longitude and latitude of geographic coordinate system data into plane coordinates, and converting the plane coordinates into a local coordinate system of a spatial mapping grid formed by simultaneous positioning and a map construction technology; through coordinate transformation, geographic coordinate system data can be accurately mapped into a model space coordinate system of the mixed reality equipment, the limitation that external high-precision positioning data are difficult to integrate in a traditional mixed reality technology is broken through, and high-precision equipment positioning is realized; meanwhile, an initial weight value is distributed to each grid unit in the space mapping grid, the weight is dynamically adjusted to change the size of the grid, dynamic management of space mapping grid data is achieved, detailed real space mapping grid data can be reserved after initial alignment is carried out through the space mapping capacity of mixed reality equipment, and the space mapping capacity of the mixed reality equipment is improved. And a data basis is provided for subsequent dynamic collision detection.

The invention is applicable to the technical field of medical image processing and mixed reality, and provides a laparoscopic surgery mixed reality navigation method based on deep learning and dynamic point tracking, which comprises the following steps: dynamically registering a three-dimensional model containing kidney, tumor and vessel with an initial frame of a laparoscope through a mixed reality alignment technology; the method comprises the following steps: constructing an operating forceps motion sensing model based on a time sequence deep neural network, realizing real-time control and parameter locking of a three-dimensional model pose, dynamically updating a two-dimensional feature point set by adopting a multi-feature-point combined tracker, constructing a candidate feature combination through a cross-quadrant sampling strategy, and constructing an operating forceps motion sensing model; a candidate feature combination is generated through a four-quadrant division and cross-regional sampling strategy, an optimal camera pose parameter is generated in combination with a re-projection error and pose continuity constraint, and an operation video is dynamically covered with a semitransparent three-dimensional model. The method can significantly enhance the spatial perception capability of the kidney anatomical structure, reduce the registration error of the kidney in the three-dimensional integrated kidney structure model and the laparoscope video, and improve the navigation precision.

A computer-implemented method for optimizing for virtual content occlusion in mixed reality may include (i) receiving a frame of a passthrough video captured by a sensor of a mixed-reality display, (ii) detecting at least one region of the frame of the passthrough video that is occluded by a virtual object projected within the mixed-reality display, (iii) creating a modified version of the frame of the passthrough video without the region that is occluded by the virtual object, (iv) providing the modified version of the frame to at least one visual analysis algorithm that calculates at least one statistic based on the modified version of the frame, (v) adjusting the frame by at least one visual adjustment algorithm based at least in part on the at least one statistic, and (vi) displaying the adjusted frame on the mixed-reality display. Various other methods, systems, and computer-readable media are also disclosed.

A device for simulating a weapon in a virtual reality (VR) environment, an augmented reality (AR) environment, a mixed reality (MR) environment, and / or combinations thereof. The device is adjustable to fit a user's shape and receive a gaming controller. The device includes a forearm grip and a front-end grip that improves the simulation of weaponry and user immersion.

The invention relates to a BIM model mobile terminal and mixed reality cooperation method and system and a medium, and the method comprises the steps: carrying out the space segmentation and space grid division processing of a BIM model according to a pre-created index structure, obtaining the index information of a plurality of space models and a plurality of space grids, and obtaining a plurality of space models and a plurality of space grids based on the index information; and constructing a unique identification code of the spatial model. Identifying the identification code through the mobile terminal, loading the spatial model corresponding to the identification code, obtaining spatial coordinates in the spatial model according to the gesture operation, converting the spatial coordinates into spatial grid index information through a preset conversion rule, and generating a loading instruction of the mixed reality equipment. And the mixed reality equipment loads a space model corresponding to the space grid index information according to the loading instruction, and superposes the space model to a real building scene through a positioning strategy. The problem of low cooperation efficiency of a BIM model mobile terminal and mixed reality in related technologies is solved.