15260results about "Graph reading" patented technology

The method can include: recording a neural state, determining a latent array, and determining an output based on the latent array. In variants, the method can function to enable a subject to use neural signals to navigate within a high-dimensional concept space to control an output (e.g., image generation, motor control, communication, etc.).

The invention discloses an AR home experience method in a large scene. On the basis of combination of a natural feature identification-based three-dimensional registration method and a binocular tracking positioning and local mapping method, the camera attitude is estimated by using feature points of a real-time scene and corresponding three-dimensional points thereof under the binocular trackingpositioning and local map construction technology. According to the mode, on-site environment features shot in real time are used as recognition tracking objects, a virtual home model can still be normally positioned and tracked under the condition that no identification graph exists, the problems that an existing AR home experience application is small in use range and poor in stability are solved, and therefore the AR home experience of virtual and real fusion can be met in a wider range and more truly.

The invention relates to the field of intelligent interaction, and discloses an intelligent hardware dynamic interaction system based on voice semantic fusion and multi-modal perception, which comprises the following steps of: constructing a context model of continuous operation by collecting continuous voice instructions, gesture actions and expression information of a user; semantic analysis and feature fusion are carried out on currently collected voice, gesture and expression features, meanwhile, credibility indexes of all modes are calculated through a weighting or deep learning model, weighting correction is carried out on a fusion result, a real-time feedback algorithm is adopted for weight adjustment for continuous optimization, the next operation intention of a user is predicted through deep learning, and the user experience is improved. And in combination with historical interaction data, online feedback and prediction errors, context management, modal weight and intention prediction strategies are adaptively optimized, and the updated strategies are used for next-round context acquisition and multi-modal fusion. The method has the advantage of improving the recognition accuracy in the continuous interaction scene.

The invention relates to the technical field of action recognition, in particular to a somatosensory action interaction recognition method and system based on skeleton coordinate points. The method comprises the following steps of collecting real-time skeleton coordinate data of a human body and performing multi-modal feature extraction to obtain a real-time skeleton coordinate sequence; obtaining a standard skeleton posture corresponding to the target interaction action, performing pre-recording and feature coding, and generating a target posture skeleton feature template library; performing skeleton time sequence filtering and joint mapping and joint included angle calculation on the real-time skeleton coordinate sequence, performing similarity measurement and dynamic binding tracking at the same time, and starting a binding recovery mechanism when binding loss is detected so as to guide the user to execute a preset binding posture and re-establish a binding relationship; and mapping the joint included angle time sequence data to a corresponding joint of the virtual human shape interaction model in real time, outputting a somatosensory interaction instruction, and driving to repeat a human body action so as to trigger a somatosensory action interaction event. According to the invention, the stability of somatosensory action interaction recognition can be improved.

The invention relates to a tunnel illumination control method considering visual adaptability of a driver, and belongs to the technical field of tunnel illumination control, and the method comprises the following steps: S1, employing a volume scattering point location method to obtain visual adaptability parameters of visual adaptation states of all segments in a tunnel; s2, dynamically constructing a predictive visual adaptation model of the driver group based on the visual adaptation parameters, and outputting expected visual states of the drivers according to the predictive visual adaptation model; s3, according to the expected visual state, a parameter control mechanism is adopted to control target illumination parameters of all the sections of the tunnel; s4, according to the target lighting parameters, the working state of lighting equipment in a corresponding section in the tunnel is regulated and controlled, so that the light environment in the tunnel is matched with the dynamic visual adaptability of a driver, and safe visual guidance is achieved; the method has the beneficial effects that the target illumination parameters of all the sections of the tunnel are controlled by adopting a parameter control mechanism according to the expected visual state according to the target illumination parameters.

Embodiments of the invention provide an active pen-stylus that emulates a paper feeling for users of the active pen-stylus. The active pen-stylus comprises a writing shaft that receives physical forces arising from use of the pen-stylus by a user. The active pen-stylus also includes a force sensor that receives forces from the writing shaft, the force sensor configured to convert received forces into an electronic signal. A writing spring receives forces from the force sensor and reflects back to the user a reactionary force that emulates a paper feeling. The writing spring compresses and undergoes geometric deflection in creating the reactionary force that emulates the paper feeling. The geometric deflection produced by the writing spring arises from deformation of the writing spring by the received forces.

The invention discloses an intelligent glasses image adjusting system based on eye movement tracking and gesture fusion, and relates to the technical field of intelligent equipment. A multi-modal sensing module is arranged to construct a multi-modal sensing layer to capture eyeball movement tracks and gesture actions; a fixation point prediction module is set to process a dynamic scene through a space-time attention mechanism to obtain a fixation point prediction area, a gesture semantic understanding module is set to process gesture actions based on a Transform architecture, and the gesture actions of a user are converted into image adjustment instructions. An image enhancement strategy setting module designs a multi-stage image enhancement strategy according to the fixation point prediction area and the image adjustment instruction, and sets a dynamic adjustment intensity control module to perform adaptive adjustment to obtain a dynamic adjustment intensity control result; an eye movement-gesture cooperative control module is arranged to provide an eye movement-gesture cooperative control mechanism to realize image area selection and parameter adjustment, and accurate image area selection and parameter adjustment are realized.

The invention discloses a robot control system and method based on vision and force sense fusion and a robot, and relates to the technical field of robots. The robot control system based on vision and force sense fusion and the control method thereof analyze visual features in real time through an artificial intelligence processing unit to predict mechanical parameters; and the self-adaptive impedance control unit is combined to dynamically fuse a position instruction and force sense feedback, so that precise adaptation of target object characteristics and compliant regulation and control of an interaction process are realized, real-time sensing and dynamic adjustment capabilities are achieved, assembly damage caused by mechanical mismatching is effectively avoided, and the reliability and efficiency of precise assembly are improved.

In some embodiments, a computer system performs different object selection-related operations. In some embodiments, a computer system places objects at locations in a displayed region based on attention of a user. In some embodiments, a computer system displays a container virtual object with curvature in a three-dimensional environment.

The invention discloses an immovable cultural relic three-dimensional reconstruction system which comprises a data acquisition module, a feature extraction module, a restoration simulation module and an optimization and decision module. The data acquisition module is used for acquiring three-dimensional data of an antique; the feature extraction module is used for extracting key features of antiques by using a deep learning algorithm; the restoration simulation module is used for constructing a virtual restoration model based on the extracted features, simulating an antique restoration process and evaluating a restoration scheme; and the optimization and decision module is used for realizing precise positioning and navigation of a repair site in combination with the SLAM technology, optimizing a repair path and assisting a repairer in making a scientific decision. According to the method, the repairing efficiency and precision are remarkably improved, automation and intelligentization of the ancient object repairing process are achieved by introducing advanced technologies such as deep learning and NeRF, the repairing efficiency and precision are greatly improved, and the repairing period is shortened.

The invention discloses a multi-modal perception and optimization method and system for a low-power-consumption AR device, and the method comprises the steps: collecting multi-modal data, task demands, resource state data and environment data for the AR device; lightweight processing is carried out on the multi-modal neural network model through model pruning, parameter quantification and distillation technologies; inputting the collected data into a lightweight multi-modal neural network model for dynamic reasoning to obtain a multi-modal recognition result; comprising the steps of executing modal adaptive weight acquisition based on task requirements and environment data; executing energy consumption constraint scheduling according to the equipment resource state data, dynamically selecting a reasoning path strategy, and obtaining corresponding modal feature output; multi-modal feature fusion is carried out, task reasoning is completed, and a multi-modal recognition result is obtained; early-leaving control is executed based on a middle-layer confidence coefficient threshold value in the reasoning process; and interactively outputting a real-time multi-mode identification result. According to the invention, energy efficiency and precision balance and multi-mode fusion low-power-consumption optimization can be realized.

A system that can deliver haptic feedback by applying an electrical stimulation to a first area of a user's body to induce a second area of the user's body to experience a level of natural sensation in response to an action occurring in a simulated remote environment and an intensity of the action is described. The system includes a controller to set parameters for the electrical stimulation based on the action occurring in the simulated remote environment and an intensity of the action. The system also includes a signal generator to generate the electrical stimulation comprising the parameters. The system also includes a skin surface electrode placed at a first location on a user's body remote from a second location on the user's body to deliver the electrical stimulation with the parameters to a nerve at or near the first area of the user's body.

The invention relates to a motor imagery ability evaluation and training enhancement system and method based on electroencephalogram and a medium, and belongs to the technical field of brain-computer interfaces. The system comprises an electroencephalogram acquisition device, a processing terminal and a display device. By collecting and analyzing electroencephalogram signals of a subject, the system extracts time-domain, frequency-domain and space-domain features by using a multi-feature fusion technology, so that accurate quantitative evaluation of motor imagery ability is realized. The evaluation core index is a lateral index. A built-in self-adaptive training module dynamically adjusts training difficulty and comprises a basic mode, a middle-level mode and a high-level mode, and personalized efficient training is ensured. The method comprises pre-training guidance, data acquisition and processing in formal training, and adaptive training adjustment based on an evaluation result. By combining multi-feature fusion and an adaptive training mechanism, an efficient and personalized solution is provided for evaluation and enhancement of motor imagery ability, so that the rehabilitation training effect of a motor imagery brain-computer interface system is more effectively improved.

The invention relates to the technical field of brain-computer interfaces, and provides a brain-computer interface instruction issuing method, device and equipment based on regulation enhancement simulation, and the method comprises the steps that an electroencephalogram decoding model comprises an encoder, a feature enhancer and a task classifier, the encoder encodes a real-time electroencephalogram signal to obtain compression representation before nerve regulation, and the feature enhancer is used for classifying the compression representation before nerve regulation; the feature enhancer performs feature enhancement on the compression representation to obtain enhanced representation, and the task classifier classifies the enhanced representation to obtain an electroencephalogram decoding result. According to the method, a feature enhancer is obtained by combining training of a state discriminator based on a sample electroencephalogram signal collected before nerve regulation and a real state label after nerve regulation, and the feature enhancer is driven to learn a feature migration relation between a compression feature before nerve regulation and a feature after nerve regulation; the feature characterization capability of an electroencephalogram decoding model on electroencephalogram signals is remarkably improved, so that the decoding robustness on weak stimulation signals is enhanced on the premise of not depending on high-intensity external stimulation.

The invention provides a neural rehabilitation training method and system integrating a brain-computer interface and virtual reality, and relates to the technical field of brain-computer interfaces. The method comprises the following steps: constructing an aligned multi-modal feature sequence by collecting electroencephalogram, myoelectricity, joint kinematics, eye movement and physiological load signals; generating an immersion parameter prescription in the baseline stage and setting a time delay and synchronization strategy; according to the nerve quality index, performing cooperative self-adaption of decoder parameters, prescriptions and peripheral assistance; establishing a drift model after the session to update the prior and shorten the re-calibration time; and monitoring dizziness and task load in real time and executing grading treatment. According to the invention, stable closed-loop individualized rehabilitation training is realized, the decoding performance and the rehabilitation effect are improved, and the safety and long-term convergence are ensured.

The invention relates to the technical field of medical rehabilitation, and discloses a vestibular function and cognitive function rehabilitation training system and method based on virtual reality, and the rehabilitation training system comprises a hardware platform, a software platform and a server. The hardware platform comprises VR interaction equipment, a head motion tracking sensor and calculation and display equipment; the software platform is integrated with a user management module, a visual stimulation module, a vestibular rehabilitation training module, a cognitive evaluation and training module, a dynamic visual acuity evaluation module and a data management and analysis module. By combining black and white chess grid visual stimulation, standardized vestibular rehabilitation tasks, spatial cognitive testing and DVA objective monitoring, synchronous evaluation and personalized collaborative intervention of vestibular functions and cognitive functions of dizzy patients are realized. According to the method, the problems of boring traditional rehabilitation means, subjective evaluation and lack of quantitative feedback are solved, and the scientificity, interestingness and curative effect testability of rehabilitation training are remarkably improved.

The invention discloses an intelligent agent digital image interaction generation method based on multi-modal perception, which comprises the following steps: collecting multi-modal input data of a user, and respectively carrying out preprocessing and feature extraction on the multi-modal input data; inputting to an improved efficient modal cross learning network, and carrying out multi-modal feature fusion processing; constructing a semantic intention map, introducing a time index edge weight and an emotion driving edge weight, and encoding the map by using a structure perception map neural network; a modal style vector is extracted through a cross-modal style contrast learning mechanism, and a personalized style coding vector is generated through a hierarchical nested structure; inputting a personalized regulation and control gating mechanism, and regulating and controlling the middle layer representation in the interaction strategy generation process by adopting a feature channel linear modulation method; inputting the representation vector into a behavior strategy generation module to generate a multi-modal behavior output sequence; and the sequence is output to drive the digital image to perform synchronous response, and natural response generation in the user interaction process is completed.

The invention discloses a large-span truss hoisting and splitting system and method based on digital twinning, and belongs to the technical field of building construction, and the system comprises a physical entity unit, a data acquisition and transmission unit, a digital twinning model unit, a function service and data analysis unit and an application and interaction unit. According to the invention, through real-time sensing of a physical entity state, multi-source data fusion transmission, high-precision digital twinning modeling, intelligent simulation analysis and decision optimization, and fusion of multi-modal man-machine interaction, a virtual control system which is in whole-course synchronous mapping and bidirectional interaction with a physical entity is constructed; according to the method, real-time visual monitoring, trend prediction and risk early warning of the hoisting and splitting process can be achieved, scheme preview and dynamic optimization are supported, virtual-real fusion operation guidance is achieved through the AR technology, and the safety, precision and efficiency of hoisting and splitting operation of the large-span truss are improved.

The invention discloses a digital large-screen interaction method and device based on multi-agent cooperation and a medium, and relates to the technical field of digital large-screen interaction.The method comprises the steps that postures, expressions and voice data of a user are collected in real time, the attention weight is calculated in combination with spatial position information, the emotional state change rate is monitored, and the user experience is improved in combination with personal historical browsing preferences; acquiring a main interaction user, an emotional state feature and an emotional change trend; according to the main interaction user, the emotional state features and the emotional change trend, obtaining an initial confidence coefficient of the suspected interested field through a semantic understanding model; and based on the candidate guide images, identifying a user selection intention through a multi-modal fusion processing mechanism, calculating a selection probability, triggering content activation of the main interaction area and content dynamic generation of the auxiliary information area, synchronizing state information, and starting an immersive collaborative interaction process. Through three-level collaborative service configuration, the beneficial effects of improving large screen content organization efficiency and enhancing immersive interaction experience are achieved.

The invention discloses a multi-scene self-adaptive man-machine interaction system and method based on emotion recognition, relates to the technical field of man-machine interaction, and solves the technical problems of realizing fusion perception of multi-modal emotion features and improving the accuracy of emotion judgment in complex scenes. According to the method, facial, voice and text emotion features are extracted by adopting a multi-modal fusion technology, the limitation of single-modal recognition is solved, an emotion-scene association rule base and a user portrait are constructed, real-time scene classification is combined, accurate mapping of emotions, scenes and demands is realized, one-step interaction of strategies is avoided, and the user experience is improved. Language interaction adaptation is designed from the form, content and style three-dimensional degree, it is ensured that languages are natural and fit scenes, functional response adaptation improves efficiency through priority ranking and execution mode optimization, environment linkage adaptation is combined with user emotion dynamic adjustment directions, collaborative linkage of languages, functions and environments is achieved, and strategy splitting is avoided.

The invention relates to the technical field of computers, and discloses a real-time rendering and interaction method and system for an immersive virtual reality scene. The method comprises the following steps: fusing tuner inertial data and eyeball tracking data, and constructing a prospective state prediction model; generating a predictive focus field in combination with scene visual saliency; synthesizing an anisotropic temporal-spatial resolution graph according to the predicted head angular velocity; gPU variable-rate coloring is driven to realize non-uniform rendering; and re-projection or dynamic fuzzy correction is executed in a self-adaptive manner according to the attitude prediction error before display. According to the technical scheme, the perception delay and the rendering load are remarkably reduced, and the frame rate stability and the visual immersion in a high-dynamic scene are improved.

The invention discloses a tourism service agent system and method based on a multi-modal large model. The system comprises five parts: a user interaction interface, which is used for receiving a user request and returning a response; the agent center is used for performing intention understanding, task planning and response generation through a multi-modal large language model; and the tool calling module is used for executing specific API business operation. According to the intelligent tourism service system, the work of each module is coordinated through the intelligent agent center, a complete closed loop from user intention understanding to service execution is realized, accurate, reliable, whole-course and personalized intelligent tourism service can be provided, and the problems that a traditional tourism service system is single in function, inaccurate in information and lack of action ability are effectively solved.

The invention relates to the technical field of auxiliary training, and discloses an AR auxiliary training system for improving the diagnosis and treatment capability of general practitioners on undifferentiated diseases, and an immersive training environment is constructed by adopting an augmented reality technology. The system comprises an environment sensing module, a space positioning module, a case data module, a virtual rendering module, an interactive acquisition module, an intelligent response module, a state control module and an evaluation calculation module. The environment sensing module adopts an RGB-D camera and an inertial measurement unit to collect environment and head movement data of a user; the spatial positioning module determines a virtual patient placement position based on an SLAM algorithm; the virtual rendering module renders a three-dimensional virtual patient in a real environment; the interaction acquisition module supports voice and gesture multi-mode interaction; the intelligent response module generates an intelligent patient answer; the evaluation calculation module carries out comprehensive evaluation from three dimensions. The invention provides a highly immersive and intelligent training environment, and has the advantages of low cost, reusability and high standardization degree.

The invention relates to the technical field of virtual reality, in particular to a garden landscape digital display and interaction system, hand track depth difference is extracted to construct a gradient path, a facade center point is combined to calculate an orientation included angle, jump calibration projection is analyzed, a leaf surface number is recognized, and an offset adjustment orientation is calculated according to a normal and an illumination included angle. And constructing a similar matrix recombination frame sequence scene, and counting an interaction picture presented by a hotspot coverage combination path access terminal. Visual inertia synchronization is realized by quantifying the depth difference and direction of a hand track, generating nodes based on a gradient threshold, smoothing a path, combining sudden change of an included angle between the nodes and a building center vector and identifying a visual angle switching area, the orientation is adjusted by cosine offset of a leaf surface normal and an illumination included angle, and the light and shadow continuity is improved through matrix reorganization mapping. The high-frequency interaction area is divided by combining the hotspot coverage frequency, and the response efficiency is improved.

The embodiment of the invention provides a mobile banking multi-mode interaction method and system based on a language user interface, and relates to the field of financial science and technology, the method comprises the following steps: obtaining input information of multiple modes input by a user, and fusing the input information of multiple modes to obtain a unified natural language instruction; inputting the unified natural language instruction into a large language model for intention classification and parameter extraction to obtain an operation intention corresponding to the unified natural language instruction and a corresponding service parameter; and executing the operation intention according to the unified natural language instruction and the service parameters. The problem that in the prior art, multi-mode input information in mobile phone banking application is mutually separated, and user intentions cannot be analyzed and understood in a collaborative mode is solved.

Some examples are directed to systems and methods for scrolling scrollable content in response to gaze-based inputs. Some examples are directed to systems and methods for scrolling scrollable content in response to gaze-based inputs and / or input provided by a respective input element. Some examples are directed to systems and methods for displaying virtual objects with an appearance that is based upon a duration of attention directed to the virtual objects. Some examples are directed to systems and methods for displaying animations of virtual objects. Some examples are directed to changing values of audio parameters based on attention of a user. Some examples are directed to changing gaze scrolling regions of content based on characteristics of the content.

The invention provides a vision-based cross-network interaction method and system, and relates to the technical field of intelligent interaction, and the method comprises the steps: obtaining a visual interaction sequence, constructing multi-modal feature representation, achieving cross-domain semantic alignment, deconstructing visual information into a hierarchical control instruction set, and transmitting the hierarchical control instruction set to a target network environment for execution after security classification. And a bidirectional mapping relation graph is constructed to realize incremental optimization. According to the method, semantic bridging between heterogeneous networks can be established, the cross-domain control precision is improved, and meanwhile safe interaction in a network isolation environment is guaranteed.

One or more computer systems switch a representation of a first content item at a docked position in an environment with a representation of a second content item in the environment in response to input, switch from displaying a first environment to displaying a second environment in response to input while displaying a representation of a first content item at a docked position, detect and respond to events corresponding to requests to move virtual content in an environment, detect and respond to events corresponding to requests to transition a mode of display of virtual content in the environment, display a first framing element concurrently with a representation of a content item having different degrees of transparency in response to detecting an input, and / or facilitate docking of a content item in an environment based on an input, and / or determine clusters of virtual objects and restore virtual objects after a reboot event.

An XR system is provided. This system captures images including images of a first hand of a user and a second hand of the user using one or more cameras. The XR system generates cropped images using the images, each cropped image including a surface of the first hand. The XR system detects a hand touch of the surface of the hand by a digit of the second hand using the cropped images. The hand touch is used as an input into an XR user interface of the XR system. The surface of the hand can be palmar surface or a hand dorsal surface.

A mobility augmentation system monitors a user's motor intent data and augments the user's mobility based on the monitored motor intent data. A machine-learned model is trained to identify an intended movement based on the monitored motor intent data. The machine-learned model may be trained based on generalized or specific motor intent data (e.g., user-specific motor intent data). A machine-learned model initially trained on generalized motor intent data may be re-trained on user-specific motor intent data such that the machine-learned model is optimized to the movements of the user. The system uses the machine-learned model to identify a difference between the user's monitored movement and target movement signals. Based on the identified difference, the system determines actuation signals to augment the user's movement. The actuation signals determined can be an adjustment to a currently applied actuation such that the system optimizes the actuation strategy during application.