55 results about "Haptic interaction" patented technology

The invention discloses an operation AR intelligent auxiliary management system based on voice AI interaction driving, which is applied to the field of intelligent warehousing and comprises an environment perception type voice interaction module, an AR multi-mode interaction compensation module, a dynamic noise robustness data processing module, an operation management module and a database module. The environment perception type voice interaction module is used for collecting voice signals of operating personnel and inhibiting environment noise; the dynamic noise robustness data processing module is used for carrying out noise reduction processing, semantic understanding and closed-loop optimization on the voice signal and generating feedback information and a control instruction; the AR multi-modal interaction compensation module is used for receiving feedback information of the dynamic noise robustness data processing module and providing visual and tactile interaction compensation, and the job management module is used for scheduling management equipment according to a control instruction; according to the method, the operation efficiency can be effectively improved, the accuracy is high, the real-time performance is high, and interaction experience optimization can be realized.

The invention discloses a cognitive impairment risk intelligent matching intervention system based on multi-modal data, and belongs to the technical field of cognitive impairment, and the cognitive impairment risk intelligent matching intervention system specifically comprises the following steps: collecting a tactile interaction sequence and a face video data stream when a user executes a cognitive intervention task, and constructing an original multi-modal data set; extracting tactile operation track features and reconstructing a heart rate variability sequence, and generating a synchronous multi-mode feature set; identifying a difference interval of state fluctuation and extracting behavior and physiological features in the interval to form an instant physical and mental load parameter set; in combination with an interaction efficiency index in the task execution record, generating a joint state feature vector, and outputting a personalized adaptation instruction through a pre-trained sub-state-parameter association model; and constructing an adaptive strategy model by using the multi-modal features and an adaptive instruction to realize millisecond-level dynamic adjustment from the real-time multi-modal features to task parameters. According to the method, real-time perception and accurate matching of the cognitive emotion load of the user are realized, and the individuation degree and instantaneity of intervention are improved.

The invention provides an intelligent building system interaction experience method and system combined with VR, and the method comprises the steps: collecting a continuous behavior sequence of a user in a physical space through a VR interaction terminal, and generating virtual-real behavior mapping data containing the association relationship between the physical behavior of the user and a virtual building scene function region; based on the virtual and real behavior mapping data, constructing a user intention evolution trajectory, covering an intention change trend of a time dimension and a region attention path of a space dimension, and generating a dynamic reconstruction instruction containing a space layout adjustment rule and an interaction element activation strategy according to the intention evolution trajectory; driving real-time spatial recombination of a virtual building scene according to the dynamic reconstruction instruction, generating a multi-modal feedback signal containing visual scene update data and tactile interaction response data, collecting a real-time response behavior of a user to the feedback signal, optimizing a prediction model of an intention evolution trajectory, and performing real-time reconstruction on the intention evolution trajectory. Therefore, the dynamic adjustment of the virtual building scene according to the intention of the user is realized, and personalized and immersive interaction experience is provided.

The invention relates to the technical field of tactile interaction, in particular to a virtual material interaction system with tactile feedback, which comprises a contact state sensing module, a potential gradient calculation module, a friction dynamic regulation and control module, a tactile feedback generation module and an interaction state monitoring module. According to the invention, dynamic mapping of contact pressure, sliding speed and node potential is realized through multi-source signal fusion, so that touch perception keeps continuous response in time sequence, potential difference calculation forms an electric field regulation basis for microscopic change of a material, and space-level adjustment of a surface friction coefficient is realized through charge redistribution and polarization direction switching. The tactile feedback changes adaptively according to material features, the friction spectrum is recombined into a multi-frequency-domain tactile signal after Fourier decomposition, texture, hardness and damping features of different materials are presented in a layered mode, the tactile feedback has more dynamic details and direction consistency, the sense of reality and the tactile recognition degree of virtual materials are improved, and the sense of reality and the sense of touch recognition degree of the virtual materials are improved. And stable and natural interaction touch in the virtual environment is ensured.

The invention belongs to the technical field of tactile communication and intelligent man-machine interaction, and discloses a multi-point tactile intelligent encoding and decoding device, system and method, and the method comprises the steps: obtaining an original tactile signal through an acquisition end, and extracting features through a neural network auto-encoder; a weighted fusion module based on an attention mechanism combines channel state information (CSI) to carry out dynamic grouping residual quantization on coding features, and low-code-rate compression transmission is realized; the sensing end inversely quantizes received data and reconstructs a signal, and outputs a tactile signal in combination with a multi-scale sensing calibration mechanism in an optimized manner, so that the tactile signal is closer to original input in a human body sensing dimension; the multi-point tactile feedback device obtains vibration feedback corresponding to the real-time multi-channel vibration tactile signal based on the reconstructed signal. By fusing the deep learning architecture and the psychological tactile perception model, efficient compression, low-delay transmission and high-quality reconstruction of tactile signals are achieved, and the method is suitable for high-precision tactile interaction scenes such as virtual reality, remote operation and medical robots.

The application discloses a flexible tactile sensing and feedback system and method based on texture recognition, which comprises an electrostatic flexible tactile sensor, a feedback actuator and a tactile feedback control circuit, the electrostatic flexible tactile sensor comprises a tactile interaction film for acquiring texture features and a pressure sensor for texture feature conversion; the feedback actuator completes tactile feedback of the texture feature signal acquired by the flexible sensor in the form of vibration; the tactile control system is mainly connected with the electrostatic flexible tactile sensor and the tactile feedback actuator through the feedback control circuit; the tactile feedback actuator has a stimulation reinforcement structure on the contact surface with the human body, which can further enhance the fine feedback of the tactile feeling. The application can effectively identify the texture features and complete the feedback of the fine tactile feeling in the form of vibration, provides effective assistance for the tactile recovery of the disabled, and has important application value in the game industry, especially in the development of virtual reality technology.

This invention discloses a force feedback method, device, electronic device, and storage medium. The method includes: determining a target surgical instrument; during a simulated surgery performed on a skin tissue model using the target surgical instrument, determining a first real-time position of the pierceable tissue entity at a target time point, and determining a first reference position of the moving entity on the tissue at a reference time point corresponding to the target time point; determining a second real-time position of the moving entity on the tissue at the target time point based on the first real-time position information and the first reference position information; determining a target feedback force of the target surgical instrument at the target time point based on the first and second real-time position information; and providing force feedback to the object operating the target surgical instrument based on the target feedback force. Based on this invention, force feedback-based simulated surgery can achieve a tactile interactive perception that more closely resembles actual surgery.

An immersive scene description is defined by a data structure containing information representing triggers and haptic effects. When a trigger occurs while the user is interacting with the immersive scene, parameters for the haptic effect are retrieved from the data structure and provided to the haptic actuator for rendering the haptic effect. These parameters determine at least the modality and perception of the haptic effect, and can determine at least some of the body parts to which the haptic effect should be applied. The modality relates to the type of haptic effect, such as temperature, vibration, pressure, or acceleration. Perception determines the haptic signal to be applied to the corresponding haptic actuator. This signal is generated or retrieved from a file or other device.

The application discloses a non-contact force haptic reproduction system and method based on three-dimensional orthogonal background electromagnet, which realizes force haptic reproduction in a real operation space in a three-dimensional orthogonal background electromagnet through a three-axis fingertip electromagnet worn on a fingertip, wherein the fingertip electromagnet is composed of sub-modules, and the background electromagnet is composed of sub-modules; simulation obtains influence of force of the background electromagnet on the fingertip electromagnet, and obtains a mapping relationship of the force of the fingertip electromagnet to the background electromagnet excitation current in the whole three-dimensional operation space; three-dimensional position coordinates and rotation angles of a human finger are obtained by an optical method, and are sent into a force haptic interaction model to calculate target three-dimensional feedback force; based on offline simulation data and given three-dimensional position coordinates, rotation angles and target three-dimensional feedback force of the human finger, excitation currents of the background electromagnet and the fingertip electromagnet are finally determined.

A system for managing interactive communication includes user devices linked to users, member devices linked to members, an interactive user interface module coupled with the user devices, and a content management module coupled with the member devices. A communication module establishes network communication among the user devices, member devices, interactive user interface module, and content management module. Member devices are configured to pair with user devices. The content management module handles various interactive content types to be displayed by the interactive user interface module on user devices, either automatically or in response to user haptic interactions. Further, a method for managing communication is also provided.

The invention discloses a rehabilitation robot auxiliary control system and method based on multi-modal perception, and belongs to the field of robot auxiliary rehabilitation and mobile support. The method comprises the following steps: acquiring and processing tactile interaction information between a user and a robot, and calculating contact force, torque and a contact position; in order to ensure stable and reliable support, centroid dynamics is used for modeling a human body state, and nonlinear model predictive control is used for optimizing a sequence of contact force and a contact position; inputting the contact force, the torque and the contact position into an admittance controller, generating a track adjustment amount, and performing adaptive correction on the reference track; solving a joint angular velocity control instruction of the robot by adopting a quadratic programming method; the joint position, the speed limit and the self-collision avoidance constraint are strictly included in the quadratic programming problem, and the safety of the whole interaction process is ensured. The walking ability can be remarkably improved, emergency situations such as falling down can be effectively handled, and the walking aid is particularly suitable for the field of elderly nursing.

A haptic interaction information processing method, device, and apparatus are disclosed, which belong to the field of haptic signal representation. The haptic interaction information processing method according to an embodiment of the present disclosure includes: counting the number of waveforms of haptic interaction signals in a target video screen, where the target video screen is one frame video screen in the target video, and one waveform of haptic interaction signal corresponds to the haptic interaction information of one video screen area in the target video screen (S101); and generating a grayscale information representation diagram corresponding to the target video screen based on the number of waveforms of the haptic interaction signals, where the grayscale information representation diagram is for representing the distribution position of each video screen area in the target video screen (S102).

The invention relates to the technical field of medical rehabilitation equipment control, in particular to a leg fracture bedridden complete-cycle rehabilitation system based on hybrid real force touch interaction and a control method, by adopting the method provided by the invention, a multi-mode rehabilitation state feature set is constructed, a coupled feature set is formed through feature coupling, and the rehabilitation effect of a patient is improved. The method comprises the following steps: firstly, dynamically updating a local dynamic transition threshold value and a global dynamic transition threshold value, setting a trigger condition, judging whether to transition from a current rehabilitation stage to a next stage or not according to the trigger condition, and dynamically updating the local dynamic transition threshold value and a global accumulation threshold value; and a control signal for virtual display is sent to the MR interaction unit according to the current rehabilitation stage and a preset condition, so that real-time adaptive switching of multi-stage rehabilitation training is realized, a mixed reality function is realized in combination with the MR interaction unit, lightweight, immersive and personalized rehabilitation experience is provided for a patient, leg function recovery is accelerated, and the rehabilitation effect of the patient is improved. The rehabilitation efficiency and quality are improved.

The application discloses a feedback method of a social robot based on tactile interaction and the social robot, and the feedback method comprises the following steps: a sensing module acquires petting data of a visitor and sends the petting data to a control chip, a tactile feedback module and a motion simulation module; the tactile feedback module provides tactile feedback to the visitor according to the petting data, and the motion simulation module performs initial simulation motion according to the petting data; the control chip classifies the petting data to obtain current petting classification and sends the current petting classification to the tactile feedback module and the motion simulation module; and the motion simulation module adjusts the initial simulation motion according to the current petting classification to provide visual feedback to the visitor. The application does not need to occupy the attention of the visitor and can reduce the consultation pressure of the first-time visitor.

The application provides a natural tactile interaction method and component of a data-driven bionic robot, and the method comprises the following steps: collecting tactile interaction process data of a real organism corresponding to the bionic robot and a user and statistically aggregating the data to obtain a training data sequence; establishing a data-driven interaction prediction model and training the data-driven interaction prediction model by using the training data sequence; determining a tactile input signal generated by a tactile input system of the bionic robot acted on by the user; and controlling the natural tactile interaction of the bionic robot according to the tactile input signal by using the trained data-driven interaction prediction model. By the method of the application, the touch action of the user can be recognized by the tactile input system, the natural feedback that the bionic robot should give to the user can be calculated by the data-driven interaction prediction model, and the closed loop of the natural tactile interaction feedback can be realized by controlling the action of the bionic robot.

The application relates to the technical field of virtual reality, in particular to a VR tactile feedback system based on multi-modal perception fusion and a method thereof, which comprises a tactile interaction terminal, a distributed tactile feedback terminal, a distributed signal acquisition unit, a computer system, a tactile presentation device, a visual device and an auditory device; the core of the application is a tactile prediction unit in the distributed tactile feedback terminal; a 3D content tactile prediction algorithm based on matrix theory is constructed; tactile texture matrix and tactile material matrix are used to realize accurate mapping of visual features and tactile features; a high-dimensional matrix is innovatively used to represent a tactile feature space; a matrix spectrum analysis technology is used to realize adaptive adjustment of tactile features; a multi-scale matrix decomposition and fusion algorithm is used to realize tactile texture reconstruction from a macroscopic view to a microscopic view; the application solves the problems of limited tactile feature expression capability, lack of adaptability and insufficient multi-modal perception cooperation in a traditional VR tactile feedback system.

The invention relates to the technical field of psychological intervention and tactile interaction, in particular to a group psychological sand table somatosensory interaction device based on tactile interaction, which comprises a pressure dynamic fingerprint acquisition unit and a dynamic adjustment feedback unit. A pressure dynamic fingerprint containing a pressure topological structure variation index and a pressure gravity center migration acceleration is obtained, a dynamic adjustment feedback unit inputs the pressure dynamic fingerprint into a lightweight space-time diagram convolutional network optimized by adversarial training, synchronously outputs an operator identity label and an operation intensity level, activates a corresponding wrist strap tactile feedback device, dynamically adjusts parameters, and outputs a corresponding wrist strap tactile feedback result. When the intensity is low, high-frequency micro-vibration is triggered, when the intensity is high, progressive pressure waves are triggered, when the similarity of two sand tool pressure fingerprints exceeds a threshold value, freezing is carried out, then a sand tool feedback channel is operated, and the equipment improves feedback accuracy and interaction orderliness and is suitable for group psychological sand table treatment.

The present application relates to the technical field of wearable terminal surface microstructure and tactile interaction, and discloses a wearable terminal surface structure forming method and a wearable terminal, the method comprising: obtaining target surface information of the wearable terminal, the target surface comprising at least one of an interaction area and a wearing contact area; establishing a microtexture database, storing texture parameters, friction characteristics and tactile distinguishability indexes of candidate microtexture structures under dry, aqueous wetting and oily wetting working conditions; dividing functional partitions according to the functional attributes of the target surface, and determining corresponding microtexture structures for each functional partition based on the microtexture database, so that the target surface forms a tactile resistance distribution and / or a friction regulation distribution; the microtexture database and texture layout rules are updated through prototype verification data backflow, so that the tactile resistance gradient corresponds to the continuous parameter risk interval, and the synergistic optimization of wearing stability, comfort and continuous parameter adjustment safety prompt is realized.

The invention belongs to the technical field of man-machine interaction and force sense feedback, and particularly relates to a force sense control method for a virtual orthopedic surgery system, which comprises the following steps of: acquiring a displacement signal generated when an operator operates a force feedback master hand; tool stress in the virtual environment is calculated through a virtual model based on the displacement signal; acquiring an actual stress signal fed back by the force feedback master manipulator; calculating a force control error between the tool stress and the actual stress; based on the error, a sliding mode controller is adopted to generate a main control quantity; meanwhile, a disturbance observer is adopted to observe the lumped disturbance in the force feedback master hand system in real time, and a disturbance estimation value is obtained; performing low-pass filtering processing on the estimated value to generate an interference compensation amount; and finally, adding the main control quantity and the interference compensation quantity, synthesizing a final control quantity, and outputting the final control quantity to the force feedback master manipulator. Through combination of sliding mode control and observer-based feed-forward compensation, the authenticity and stability of haptic interaction of the virtual operation are remarkably improved.

The invention provides a student practice effect feedback method and system based on multi-dimensional data driving, and relates to the technical field of training feedback systems. According to the method, the expert standard action library is constructed, the joint point activity frequency is dynamically analyzed based on project types, core and secondary capture points are intelligently divided, and the collected data are subjected to grading processing in combination with a weighting mechanism, so that the system efficiency is improved while the key action information precision is guaranteed; a difference value is calculated by performing weighted comparison on an action track of a student and a standard track, layered feedback is generated according to a preset threshold value, differentiated tactile prompts are output through wearable equipment, and closed-loop control from data acquisition and intelligent evaluation to real-time feedback is realized; according to the whole scheme, the accuracy and scientificity of action evaluation are improved, students are stimulated and helped to quickly correct errors and consolidate correct actions in a non-intrusive real-time tactile interaction mode, and the training efficiency, concentration degree and autonomy are remarkably improved.

The invention discloses a volume three-dimensional display system and method based on sound field assembly and photoluminescence nanoparticles. The system comprises a sound field assembly module, an optical excitation module, an aerosol management module, a sensing and interaction module and a central control core. The method comprises the following steps: firstly, generating a programmable sound field in a free space by using an ultrasonic phased array, and accurately assembling nanoparticles into an intangible three-dimensional voxel matrix; and then, the nanoparticles in the matrix are directly excited through multi-band laser scanning, so that the nanoparticles serve as self-luminous voxels to emit light, and a full-color volumetric three-dimensional image is actively rendered. According to the invention, a traditional reflective projection screen is completely abandoned, true three-dimensional and omnidirectional visible display is realized, accurate force touch interaction can be realized through sound field distortion monitoring, and the key bottleneck of the existing aerial imaging technology is solved.

The invention relates to the technical field of man-machine interaction and intelligent control, in particular to a flexible somatosensory interaction system and method based on intention probability mapping. Through setting a flexible somatosensory acquisition module, an intention modeling module, a flexible-rigid mapping field construction module, a dynamic intention screening module and an execution control module, multi-modal somatosensory information of an operator is acquired, a probability distribution model of an action intention is established, and a flexible-rigid mapping field is constructed based on an information retention and minimum semantic loss principle. And the continuous action space of the human body is converted into a limited control instruction set which can be executed by the mechanical arm. The system dynamically screens an optimal control instruction by using a confidence coefficient driving mechanism under multiple candidate intentions, and realizes accurate response of a mechanical arm to the action of an operator through closed-loop feedback. The problems that in the prior art, flexible action and rigid control freedom degrees are not matched, and intention recognition is fuzzy are effectively solved.

The application relates to a kind of haptic perception analysis method and system based on tensor decomposition and time convolution network;Method includes: using the flexible tactile sensor covering whole hand and corresponding multi-channel pressure detection circuit to collect multidimensional dynamic hand pressure tactile data;Tensor decomposition is carried out on multidimensional dynamic hand pressure tactile data, and low-order tensor data is obtained;The tensor data is used to represent the haptic interaction process when hand and object interact;The tensor data is input into time convolution network to identify the haptic interaction process and output hand pressure distribution chart.The application realizes the accuracy identification of hand dynamic haptic interaction task.

The invention belongs to the technical field of medical intelligent auxiliary systems, and particularly relates to a haptic editing system capable of being mixed with a quick virtual editing function of a real surgery, which comprises a data acquisition module used for acquiring multi-source volume data of a human body, reconstructing a digital three-dimensional model containing a bony structure, soft tissue and vascular nerves based on the multi-source data, and outputting the reconstructed digital three-dimensional model; in combination with a mixed reality space mapping technology, generating a virtual surgery scene with virtual-real fusion; the interaction control module is used for carrying out organization layering and deformation parameter space construction on the virtual operation scene data to obtain three-dimensional operation scene analysis data; and the application output module is used for analyzing the three-dimensional operation scene analysis data and the operation interaction mechanics data to generate a standard degree evaluation report and risk early warning information. According to the system, a brand-new virtual operation training platform is provided for operation workers through the mixed reality (MR) technology and the haptic interaction technology.

The immersive scene description is defined by a data structure that includes information representing triggers and haptic effects. When a trigger occurs when a user interacts with the immersive scene, a parameter of the haptic effect is obtained from the data structure and provided to a haptic actuator for rendering the haptic effect. These parameters determine at least the modality and perception of the haptic effect and may determine at least a portion of the body on which the haptic effect should be applied. The modality relates to the type of haptic effect, such as temperature, vibrotactile, pressure, acceleration, etc. A haptic signal to be applied to a corresponding haptic actuator is perceived and determined. The signal may be generated or obtained from a file or from another device.

The present application relates to a kind of paper currency digital anti-counterfeiting method based on multi-element fusion haptic reproduction, belong to the cross field of information security and man-machine interaction.It includes the extraction of haptic feature, the haptic feature " formula " is adopted by BP neural network model, the decryption of user, the rendering model information and formula information obtained are sent to haptic feature rendering unit, the weight in formula information is distributed to rendering model information, the rendering model information with weight is obtained, then the rendering model information with weight obtained is sent to signal driving unit, the haptic driving signal that can realize haptic reproduction in haptic interaction interface is obtained, the function of haptic reproduction is realized, to realize the multi-element fusion haptic anti-counterfeiting.The beneficial effect is to propose a kind of " formula " based on shape, texture, temperature and other haptic fusion, give the most real naked finger haptic feedback, can be applied in the anti-counterfeiting of digital renminbi, the authenticity of digital album, the authenticity of digital ticket and other directions.

The application discloses a human-computer interaction system and method of an automatic driving vehicle, and the human-computer interaction system comprises a first interaction module; the first interaction module comprises a first tactile interaction module, the first tactile interaction module comprises a plurality of independent interaction sub-modules, each interaction sub-module is in corresponding motion according to a motion state or driving environment information of the vehicle; and the state combinations of the plurality of interaction sub-modules are different under different motion states or driving environment information of the vehicle. The first tactile interaction module of the application is provided with a plurality of independent interaction sub-modules, different state combinations of the plurality of interaction sub-modules are used to express different motion states or driving environment information of the vehicle, and the expression of complex vehicle driving scene information is met.

The application discloses a dynamic resolution ultrasonic phased array haptic generation device and a control method, relates to the technical field of ultrasonic haptic feedback, and realizes the software adjustment of the ultrasonic haptic focus size through an EDM beam forming algorithm, thereby breaking through the limitation of the traditional DAS method which is limited by hardware. Users can dynamically select a high resolution mode or a high radiation force mode according to specific application scenarios, and the flexibility and reality of haptic interaction are significantly improved. The system structure is compact, all adjustments are completed through software, hardware does not need to be changed, and the system is convenient to integrate into intelligent automobile central control, AR / VR headgear, interactive exhibition stands and other equipment, so that more rich and adaptive haptic experiences are provided for human-computer interaction. Under the premise of not changing the hardware structure, the system optimizes the phases and amplitudes of all channels of the ultrasonic array through the software algorithm of the upper computer, and cooperates with the hardware logic control of the FPGA controller to realize the dynamic control of the spatial resolution of the haptic focus and the generation of multiple foci.

Implementations set forth herein relate to facilitating control of an automated assistant or other application by rendering an ultrasonic signal that can be modulation through tactile interaction with physical portions of a computing device. The computing device can provide an acoustic signal via a speaker of the computing device, and the acoustic signal can be received by one or more microphones of the computing device. During transmission, a portion of the acoustic signal can travel through a housing of the computing device, which the user can tap or otherwise use to provide a gesture. Audio data generated using the microphone can reflect changes to the acoustic signal, and those changes can be indicative of a particular gesture received at the housing. When a particular gesture is detected, the automated assistant or other application can respond to the gesture by initiating one or more particular operations.