2722 results about "Virtual reality" patented technology

A real-time system for the multimodal generation of virtual reality scenes based on artificial intelligence for the creation of immersive three-dimensional environments from natural language narratives, consisting of: a speech capture module configured to continuously record a user's spoken narrative via one or more directional microphones, preprocesses the captured signal by noise reduction and temporal alignment, and outputs a digital speech stream; A speech-to-text processing unit that is operationally coupled to the speech capture module and configured for real-time speech recognition using a continuous neural transformer model. The unit is trained to transcribe natural language utterances into structured text data while maintaining contextual continuity throughout the evolving narrative. a semantic interpretation processing unit that is communicatively linked to the speech recognition unit and configured to perform natural language understanding techniques to extract contextual entities, spatial references, temporal relationships, and object attributes from the transcribed narrative; the engine includes a large language model that is fine-tuned for spatial reasoning tasks; a scene graph generation module configured to transform the interpreted semantic data into a structured, hierarchical representation that defines nodes for identified entities and edges for corresponding relationships, with each node associated with metadata describing geometry, position, orientation, texture, and linking attributes between objects; a multimodal image-language model processor coupled with the scene graph generation module, wherein the processor is configured to retrieve, adapt, or synthesize appropriate three-dimensional elements from a pre-trained visual-lexical embedding space and align these elements with their semantic and spatial definitions derived from the scene graph; a scene assembly and rendering controller configured to create a cohesive virtual scene from the aligned assets, perform real-time rendering using a GPU-accelerated ray tracing pipeline, and produce a stereoscopic visual output that corresponds to the evolving narrative; A head-mounted virtual reality visualization device connected to the rendering engine and configured to display the generated immersive environment to the user in real time. The device features motion sensors and inside-out tracking cameras to detect head and body movements, dynamically updating viewing angles and perspective within the rendered scene; and a bidirectional feedback module integrated into the head-mounted device and connected to the semantic interpretation processing unit; the module is configured to interpret corrective commands, gestures, or supplementary comments from the user to refine or modify specific scene elements without interrupting the real-time visualization; The system continuously updates the virtual scene as the narrative develops, ensuring temporal synchronization between speech input and rendered output below a defined latency threshold, thus enabling a natural, dialogic construction of complex three-dimensional virtual environments.

A system for dynamic latent space adaptation using spatiotemporal kernel context for multiscale rendering with hierarchical and Lorentzian autoencoders. The Spatiotemporal Kernel Estimator (SKE) analyzes media through motion field, temporal recurrence, frequency band, and scene semantics analyzers to generate adaptive kernel parameters encoding content-specific importance distributions. The system dynamically adapts latent manifold geometry by modifying metric tensor properties according to kernel context, enabling content-aware compression that allocates representational capacity based on visual significance. A multiscale cache implements kernel-adaptive retention policies prioritizing important regions. An adaptive renderer provides intelligent level-of-detail selection based on zoom level and kernel-estimated importance, optimizing processing allocation. The self-optimizing architecture continuously refines kernel context and geometric adaptation based on user interaction and performance feedback, achieving superior compression ratios and perceptual quality. Applications include bandwidth-efficient video streaming, virtual reality, scientific visualization, and cognitive video analytics requiring intelligent context-aware visual processing.

The invention discloses a single-view large-scale outdoor scene three-dimensional reconstruction method based on three-dimensional Gaussian splashing, and the method comprises the steps: collecting a pseudo aerial image, and constructing a panoramic multi-mode supervision end-to-end single-view three-dimensional reconstruction model; meanwhile, panoramic consistency supervision, semantic constraint depth regularization and a radial weighted luminosity loss and Gaussian cutting mechanism are introduced, so that the defect of insufficient geometric constraint of traditional single-view three-dimensional reconstruction is effectively overcome, and high-efficiency and high-fidelity three-dimensional modeling of a large-scale outdoor scene under single image input is realized; the method is suitable for various actual scenes such as smart city construction, automatic driving simulation, virtual reality / augmented reality, digital twinning and the like.

The invention relates to the technical field of multi-modal posture recognition, and relates to a multi-modal posture recognition system based on Transform fusion, which fully excavates the complementary advantages of visual information in spatial detail representation and inertial information in time sequence dynamic capture through space-time alignment of multi-modal data and deep fusion based on an attention mechanism. The accuracy and the stability of attitude estimation under the conditions of visual occlusion, rapid movement and complex illumination are obviously improved; posture optimization is carried out by introducing physical constraints such as bone length constancy and joint movement range limitation, and time domain smoothing and contact state correction are applied, so that the precision of the generated three-dimensional posture sequence is ensured, a solid technical foundation is laid for improving the naturalness, safety and intelligent level of man-machine interaction, and the method is suitable for popularization and application. And meanwhile, reliable application and deep development of the intelligent robot in key fields of service robots, virtual reality, remote cooperation and the like are powerfully promoted.

The invention discloses a disaster emergency rescue nursing skill training system based on virtual reality, and relates to the technical field of virtual reality. A high-simulation virtual disaster scene is constructed based on real data, diversified injury condition virtual objects matched with the scene are generated, a secondary disaster risk diffusion effect and emergency prompt information are overlaid in real time, physiological state changes such as blood pressure and respiration generated by nursing operation differences in the rescue process of wounded personnel are dynamically simulated, and the rescue effect of the wounded personnel is improved. The problems of single scene and fixed injury condition in traditional training are solved, real-time and multi-modal feedback of student operation is realized through accurate comparison and hierarchical identification of multi-dimensional operation characteristics, the defects of guidance lagging and general are overcome, a personalized and adaptive training scheme is generated according to a comprehensive evaluation result, the progress is dynamically adjusted, and the training efficiency is improved. And scene simulation, operation practice, accurate feedback and personalized improvement are integrated.

The invention relates to the technical field of virtual reality, in particular to an indoor structure reconstruction method based on a panoramic image scene understanding algorithm, and the method comprises the steps: S11, obtaining a plurality of equidistant columnar projection panoramic images of a current indoor scene through a panoramic camera or a panoramic image splicing algorithm; s12, performing semantic segmentation on the collected panoramic image by using SAM, deducing indoor ceiling, floor and wall areas of the panoramic image on the basis of a semantic segmentation result, and generating a multi-channel semantic graph; and S13, inputting the acquired RGB panoramic images and the multi-channel semantic map into a pre-trained panoramic image depth estimation model to obtain a depth map corresponding to each panoramic image. The method is used for automatically generating an indoor structure model, the scheme comprises the core steps of semantic segmentation and layout reasoning, depth estimation and point cloud construction, structure optimization and indoor model reconstruction and the like, and a complete indoor panoramic image scene understanding scheme is formed.

Disclosed is a image processing apparatus and a method for controlling the image processing apparatus. The image processing apparatus according to an embodiment of the present disclosure may identify an object from an acquired image, determine whether the object is hidden by another object by using an aspect ratio of a bounding box of the detected object, and based on the object being hidden, estimate an entire length of the object based on coordinate information of the bounding box. Accordingly, the size information of the hidden object may be efficiently identified while a large amount of database is applied or resources of the apparatus is minimized. The present disclosure may be in connection with a surveillance camera, an automotive driving vehicle, an artificial intelligence module of at least one of a user terminal or a server, a robot, an augmented reality (AR) device, a virtual reality (VR) device, a device related to a 5G service, and the like.

The invention provides a neural rehabilitation training method and system integrating a brain-computer interface and virtual reality, and relates to the technical field of brain-computer interfaces. The method comprises the following steps: constructing an aligned multi-modal feature sequence by collecting electroencephalogram, myoelectricity, joint kinematics, eye movement and physiological load signals; generating an immersion parameter prescription in the baseline stage and setting a time delay and synchronization strategy; according to the nerve quality index, performing cooperative self-adaption of decoder parameters, prescriptions and peripheral assistance; establishing a drift model after the session to update the prior and shorten the re-calibration time; and monitoring dizziness and task load in real time and executing grading treatment. According to the invention, stable closed-loop individualized rehabilitation training is realized, the decoding performance and the rehabilitation effect are improved, and the safety and long-term convergence are ensured.

The invention relates to the technical field of medical rehabilitation, and discloses a vestibular function and cognitive function rehabilitation training system and method based on virtual reality, and the rehabilitation training system comprises a hardware platform, a software platform and a server. The hardware platform comprises VR interaction equipment, a head motion tracking sensor and calculation and display equipment; the software platform is integrated with a user management module, a visual stimulation module, a vestibular rehabilitation training module, a cognitive evaluation and training module, a dynamic visual acuity evaluation module and a data management and analysis module. By combining black and white chess grid visual stimulation, standardized vestibular rehabilitation tasks, spatial cognitive testing and DVA objective monitoring, synchronous evaluation and personalized collaborative intervention of vestibular functions and cognitive functions of dizzy patients are realized. According to the method, the problems of boring traditional rehabilitation means, subjective evaluation and lack of quantitative feedback are solved, and the scientificity, interestingness and curative effect testability of rehabilitation training are remarkably improved.

The invention relates to the technical field of data processing, in particular to an integrated multi-mode culture resource intelligent data governance and management system. The system comprises a data acquisition module, an intelligent processing module, a data management module, a knowledge organization module and an interactive display module. Synchronous acquisition of videos, audios, images and three-dimensional point clouds is realized through camera equipment, audio acquisition equipment and a laser scanner, and cultural elements are extracted through action recognition, speech recognition model processing dialect transcription, image semantic segmentation and named entity recognition by adopting a convolutional neural network. The system realizes structured description and semantic mapping of data on the basis of non-abandoned domain ontology, and constructs a knowledge graph by means of a graph database and a graph neural network. And finally, realizing three-dimensional visual display of the cultural resources by combining a virtual reality technology. According to the method, the problems of non-abandoned multi-modal data processing splitting, insufficient semantic organization and weak display interactivity in the prior art are solved, and the digital governance level and the propagation capability of cultural resources are improved.

The invention relates to the technical field of computers, and discloses a real-time rendering and interaction method and system for an immersive virtual reality scene. The method comprises the following steps: fusing tuner inertial data and eyeball tracking data, and constructing a prospective state prediction model; generating a predictive focus field in combination with scene visual saliency; synthesizing an anisotropic temporal-spatial resolution graph according to the predicted head angular velocity; gPU variable-rate coloring is driven to realize non-uniform rendering; and re-projection or dynamic fuzzy correction is executed in a self-adaptive manner according to the attitude prediction error before display. According to the technical scheme, the perception delay and the rendering load are remarkably reduced, and the frame rate stability and the visual immersion in a high-dynamic scene are improved.

The invention belongs to the field of medical artificial intelligence, and particularly relates to a multi-modal data acquisition and fusion method for Alzheimer's disease. The method comprises the following steps: firstly, synchronously acquiring eye movement, expression, voice, gait and grip strength data of a subject through a virtual reality multi-task normal form, and combining with an MoCA scale to score a result; then preprocessing and feature extraction are carried out on each modal data, and unified feature representation is constructed; on the basis, a cross-modal attention mechanism is adopted to realize interaction and weighted fusion of multi-modal features, and a unified fusion feature vector table is generated; and finally, outputting structured data organized according to task fragments for auxiliary evaluation and modeling of cognitive impairment. The method can effectively solve the problems that in the prior art, single-mode information is insufficient, and multi-mode data are difficult to align and fuse, has the advantages of being low in cost, easy to popularize and high in detection accuracy, and can be widely applied to early recognition and auxiliary diagnosis of the Alzheimer's disease.

The invention relates to the technical field of artificial intelligence and virtual reality, in particular to a large model scene drilling system based on AI-VR. The system comprises a student portrait construction module, a dynamic scene generation module, a multi-source data evaluation module and a holographic guide interaction module. According to the invention, the student portrait construction module collects multi-dimensional information and constructs a detailed portrait through clustering analysis, the dynamic scene generation module generates an initial scene model, and the multi-source data evaluation module constructs an operation step dependence graph, so that the operation condition in the learning process of the student is comprehensively and accurately evaluated; the holographic guidance interaction module uses a holographic projection technology to accurately guide students to operate correctly, and meanwhile, the AI large model dynamically adjusts scene difficulty and complexity according to real-time performance of the students, so that individual requirements of the students in different learning stages are met, error sources and influences are deeply analyzed, and a quantitative and scientific basis is provided for teaching evaluation and improved learning of the students.

The invention discloses a shielding complementing method based on a continuous streetscape panoramic image, and aims to solve the problem of information loss caused by shielding of dynamic objects (such as vehicles and pedestrians) in the existing streetscape image, and the method comprises the following steps: S1, recognizing and positioning a dynamic shielding object area in a continuous streetscape panoramic image sequence; s2, providing a continuous panoramic image-oriented feature extraction and matching algorithm (FDMPano), realizing robust feature retrieval and matching in a cross-view angle, and accurately positioning a reference image region which can be complemented; and S3, carrying out feature alignment and content mapping on the sheltered area based on a matching result, and generating an unsheltered panoramic image with consistent vision. According to the method, the internal relevance of continuous streetscape data is fully utilized, the occlusion area can be automatically completed with high quality, the integrity and availability of streetscape images are remarkably improved, and the method has wide application value in the fields of automatic driving, digital cities, virtual reality and the like.

The invention relates to the technical field of virtual reality, in particular to a garden landscape digital display and interaction system, hand track depth difference is extracted to construct a gradient path, a facade center point is combined to calculate an orientation included angle, jump calibration projection is analyzed, a leaf surface number is recognized, and an offset adjustment orientation is calculated according to a normal and an illumination included angle. And constructing a similar matrix recombination frame sequence scene, and counting an interaction picture presented by a hotspot coverage combination path access terminal. Visual inertia synchronization is realized by quantifying the depth difference and direction of a hand track, generating nodes based on a gradient threshold, smoothing a path, combining sudden change of an included angle between the nodes and a building center vector and identifying a visual angle switching area, the orientation is adjusted by cosine offset of a leaf surface normal and an illumination included angle, and the light and shadow continuity is improved through matrix reorganization mapping. The high-frequency interaction area is divided by combining the hotspot coverage frequency, and the response efficiency is improved.

The invention discloses a whole-process vascular interventional operation training method and system based on a virtual reality technology, and the method comprises the steps: firstly, developing a vascular interventional mechanics and collision simulation module, and achieving the deformation simulation of a blood vessel and a guide wire through the combination of position dynamics, a region shape matching technology and a Cosseerat elastic rod model, the collision between the blood vessel and the guide wire is treated by adopting a wide-to-narrow three-layer progressive collision detection method; secondly, constructing an immersive virtual operation scene through model construction and rendering optimization; visual angle roaming, instrument operation and interface interaction are fused, and multi-mode interaction control driving intervention training is achieved; finally, based on a'module-event 'dual-drive framework, operation tasks such as anesthesia, puncture, radiography, intervention, stent release and hemostasis are sequentially connected in series through a unified state machine, and whole-process training is achieved. The accuracy and stability of blood vessel and guide wire simulation are improved, the training immersion sense is enhanced, interventional operation whole-process training is achieved, and high teaching and practical value is achieved.

The invention discloses a single view reconstruction and rendering method, which belongs to the field of view reconstruction and rendering, and comprises the following steps of: firstly, generating multi-view feature representation with strong geometric consistency from a single input image by introducing an image diffusion module of a cross attention mechanism; a point cloud reconstruction module with self-attention and cross-attention is utilized, and multi-view information is fused to reconstruct an accurate three-dimensional point cloud; and finally, constructing differentiable three-dimensional Gaussian representation based on the point cloud, rendering the differentiable three-dimensional Gaussian representation, and outputting a new view angle image, a normal map and a depth map. According to the system, a staged training strategy is adopted, and a composite loss function including multi-scale bidirectional consistency smooth loss and feature consistency loss is innovatively used for optimization. According to the method, the problems of low geometric accuracy, multi-view inconsistency, detail missing and the like in single-view reconstruction are effectively solved, and the method can be widely applied to the fields of virtual reality, digital twinning, cultural heritage digitization and the like.

The invention relates to the field of remote sensing image processing and geological disaster monitoring, in particular to a goaf three-dimensional subsidence basin reconstruction system based on multi-source remote sensing image fusion, which comprises a heterogeneous data preprocessing module, a differential geometry-based data fusion module, a three-dimensional subsidence basin reconstruction module, a subsidence dynamic monitoring module and a virtual reality interaction module, the system innovatively introduces a differential geometry theory to solve the problem of heterogeneous fusion of a high-resolution optical satellite image, an SAR satellite image and airborne remote sensing Lidar point cloud data, maps multi-source data to a unified feature space through manifold learning, calculates an adaptive fusion weight through curvature analysis, constructs a multi-scale feature correlation matrix based on a geodesic line distance, and achieves the fusion of the high-resolution optical satellite image, the SAR satellite image and the airborne remote sensing Lidar point cloud data. A high-precision fusion image is generated, three-dimensional reconstruction is carried out in combination with subsidence basin geological parameters and a physical constraint method, it is ensured that a reconstruction result conforms to an actual subsidence physical rule, dynamic monitoring and virtual reality interaction of the subsidence process are achieved, and parameter adjustment and model optimization are supported.

The invention provides a virtual reality three-dimensional space data correction method, which comprises the following steps of: firstly, acquiring an image, depth and inertia measurement data of a virtual reality terminal, and a pose, a speed and a force feedback signal of an interaction body, and carrying out time sequence alignment under a unified time reference; an interaction event is identified through collision detection, and parameters such as a contact point, a normal direction, a relative speed, a penetration depth and a contact duration are extracted. Then, physical consistency constraints including no penetration, contact stability, friction consistency, momentum conservation and energy non-increase are constructed, and geometric consistency, depth consistency and physical consistency residual functions are established in the local contact area; and solving surface normal, depth, scale, pose, calibration parameters and other corrections through a robust nonlinear optimization method so as to update the point cloud, the grid and the calibration model. According to the method, the consistency and the stability of geometric modeling and physical interaction in the virtual reality scene can be remarkably improved, and the immersion and the interaction precision are improved.

The invention belongs to the technical field of intelligent wireless sensing and radar signal processing, and particularly relates to a millimeter wave radar gesture recognition feature processing method, which is particularly suitable for scenes with environment interference (such as walking of others and static clutter), and specifically comprises the following steps: S1, preliminary filtering by a self-adaptive filter; s2, carrying out improved I CEEMDAN decomposition; s3, I MF component classification and processing; s4, signal reconstruction; the experimental result shows that the method provides a robust and efficient solution for gesture recognition, and can be widely applied to the fields of man-machine interaction, virtual reality, intelligent equipment control and the like.

The invention discloses a virtual space multi-person interaction synchronous control method oriented to an end-cloud collaborative architecture. The invention relates to a computer graphics and three-dimensional modeling technology, and discloses a cross-format lightweight and geometric consistency maintenance method based on a three-dimensional model. Through format-independent geometric representation and a self-adaptive lightweight strategy, efficient compression and precision maintenance of three-dimensional model cross-format conversion are realized. The method specifically comprises the following steps: performing format analysis and geometric feature extraction on an input model, and establishing a unified internal representation; adaptively selecting a multi-level LOD lightweight strategy based on the complexity of the model; the accuracy of key information is ensured through geometric feature keeping and topology consistency detection; the geometric consistency is dynamically maintained by combining error monitoring and an iterative correction mechanism; and generating a target format lightweight model and carrying out quality verification. According to the method, adaptive precision control, multi-level consistency maintenance and format irrelevant processing are combined, the model size and conversion errors are effectively reduced, and cross-platform compatibility and geometric fidelity are improved. The method can be widely applied to the fields of industrial design, game development, virtual reality and the like.

The invention relates to the technical field of computer graphics and virtual reality, and discloses a meta-universe social space generation method and system. The method comprises the following steps: acquiring multi-modal input data of a user and constructing an intention vector; extracting social relation characteristics based on the intention vector, and generating a multi-dimensional scene constraint condition; generating a three-dimensional virtual environment and optimizing a topological structure by adopting a generative adversarial network according to a scene constraint condition; and fusing the scene data with the user social behavior data, and outputting a multi-terminal interactive rendering result. The system comprises a data acquisition and intention modeling module, a scene constraint mapping and environment generation module and a fusion rendering and interactive output module. According to the invention, multi-user immersive social space interaction can be realized on the VR head-mounted display, the AR glasses and the mobile terminal, and the method has the advantages of low delay, high simulation degree and cross-platform synchronization.

The invention discloses an intensive rehabilitation treatment decision method based on virtual reality, and relates to the field of medical monitoring, and the method comprises the steps: collecting user physiological state parameters in real time, creating a cloud database, and storing the user physiological state parameters collected in real time based on the cloud database; according to the method, the user physiological state parameters stored in the cloud database are traversed, the wooden barrel model is created based on the user physiological state parameters, and the standard wooden barrel model is synchronously created. The dynamic wooden barrel model is created by collecting the physiological parameters in real time, and accurate comparison with the standard model is achieved in a normalization processing and three-dimensional space alignment mode; the communication area and the risk area can be visually recognized, the abstract limitation of traditional data monitoring is broken through, and dynamic and accurate allocation of medical care resources is achieved in combination with a mobile equipment distance scheduling mechanism.

The invention relates to the technical field of virtual reality, in particular to a virtual reality-based smart travel big data supervision platform, which comprises a data acquisition layer for acquiring multi-source data, a data processing layer for preprocessing the multi-source data to form a data packet, and a data transmission layer for transmitting the data packet, acquiring a data packet and constructing a virtual reality layer of the three-dimensional virtual mirror image space; and the supervision application layer is used for supervising the three-dimensional virtual mirror image space, and respectively issuing a priority transmission instruction of data at the problem to the data transmission layer and issuing a real-time reconstruction instruction of a three-dimensional scene at the problem to the virtual reality layer when the problem is monitored. According to the invention, in the data transmission layer, the redundant data shell of the protection data packet is dynamically adjusted according to the network quality, the temporary dedicated channel is constructed based on the SDN, and the data of the problem area directly reaches the virtual reality layer, so that the high-fidelity restoration and supervision of the virtual scene in the weak network environment are realized.

The invention relates to the technical field of virtual reality experiment teaching, and discloses an experiment teaching simulation method and system based on virtual reality. The method comprises the steps that a continuous behavior sequence generated by user operation is collected, and a teaching behavior chain is generated through discretization; constructing a state vector by identifying the space-time state of the behavior unit, and comparing the state vector with a standard template to mark an abnormal behavior unit; tracing the context to generate an abnormal behavior description block, and querying an error knowledge base to obtain error information; meanwhile, analyzing a state change event of the virtual experiment equipment; and finally, fusing the error information and the state event to generate a simulation teaching evaluation result. According to the method, refined multi-dimensional analysis of experimental operation behaviors and intelligent error influence quantitative evaluation are realized, and the accuracy and guidance of teaching feedback are improved.

The invention relates to the technical field of virtual reality and indoor and outdoor design, in particular to a VR indoor and outdoor design virtual material selection system based on eye movement tracking. Comprising a VR display and eye movement tracking module used for presenting a virtual design space to a user and capturing eye movement data of the user in real time; the fixation point recognition and target object judgment module is used for calculating real-time fixation point coordinates of the user based on the eye movement data; the dynamic material menu management module is used for outputting the target object information according to the judgment module; the sight line interaction and material rendering module is used for receiving a selection instruction of a user on a specific material in the dynamic material menu through a sight line; and the central processing module is used for coordinating and controlling data streams and execution logics of the modules. By integrating the eye movement tracking technology and the dynamic material menu management, the intelligent material selection interaction driven by the fixation point is realized.

The invention discloses a dynamic rendering method and system for virtual reality, a storage medium and electronic equipment, and relates to the technical field of virtual reality, augmented reality and computer graphics. Therefore, the frequency and time sequence change of the sight focus of the user hitting the surface of the virtual object in the virtual scene are analyzed, so that the relative motion trend and observation intention of the user and the object are pre-judged, predictive and adaptive scheduling of rendering resources is realized, and optical feedback conforming to the physical law, namely, dynamic material, is triggered at the sight focus, so that the real-time performance of the user is improved. Therefore, the interaction fineness in the process of interaction between the user and the VR system is improved.

The invention belongs to the technical field of mobile edge computing, and discloses a task scheduling method based on dynamic cloud edge collaboration. According to the invention, through link-level modeling and graph neural network feature extraction, adaptive expansion of different numbers of terminals and servers is realized; according to the method, the problems of instability and over-estimation in training are effectively avoided by combining cutting updating of a near-end strategy optimization algorithm and a generalized advantage estimation method. According to the method, the elastic computing power of the cloud and the edge low-delay characteristic can be fully utilized, the dynamically changing network scale and terminal requirements can be adapted, flexible unloading of tasks and dynamic resource allocation can be realized, task processing delay can be effectively reduced in a dynamic environment, the resource utilization rate is improved, and the resource utilization rate is increased. And good expansibility and stability are maintained under the condition that the number of terminals and the scale of the server fluctuate, the problems of computing resource bottleneck and task scheduling stiffness existing in the virtual reality video service of the existing mobile edge computing are solved, and the interaction experience of the user is remarkably improved.

The invention discloses an interaction method and system based on multi-modal data, and relates to the technical field of man-machine interaction. The method comprises the steps that initial interaction data of a user is acquired, preprocessing and feature coding are carried out on the initial interaction data, target interaction data are obtained, and the initial interaction data comprise a hand depth image, a voice signal, a through hole center coordinate and a head rotation matrix; determining a target weight of each piece of target interaction data by using a preset weight strategy, performing conflict resolution on each piece of target interaction data based on the target weight of each piece of target interaction data or the timestamp of each piece of target interaction data, and fusing each piece of target interaction data after conflict resolution to obtain a target interaction instruction; and executing the target interaction instruction, and displaying the virtual scene and the operation target after executing the target interaction instruction. Therefore, the problems that the interaction dimension of the current virtual reality technology is single, the multi-modal signal fusion degree is low, a cooperation mechanism is missing, and misjudgment is easily caused are solved.

The invention provides a railway engineering information model lightweight management method and system, a storage medium and a computer program product. The method comprises the following steps: receiving uploaded original railway engineering information models from different sources; standardizing the model data format and the model detail level of the original railway engineering information model to obtain a standardized railway engineering information model; reducing the geometric complexity of each component contained in the standardized railway engineering information model by using a geometric simplification mode, and simplifying attribute information contained in the standardized railway engineering information model according to a preset standard to obtain a lightweight railway engineering information model; wherein the geometric simplification mode comprises the step of replacing geometric shapes contained in all the components with regular voxels; and carrying out visual presentation on the lightweight railway engineering information model by utilizing a virtual reality-based mode. According to the invention, lightweight management of the BIM model can be realized.