72 results about "Haptic interaction" patented technology

The invention discloses an operation AR intelligent auxiliary management system based on voice AI interaction driving, which is applied to the field of intelligent warehousing and comprises an environment perception type voice interaction module, an AR multi-mode interaction compensation module, a dynamic noise robustness data processing module, an operation management module and a database module. The environment perception type voice interaction module is used for collecting voice signals of operating personnel and inhibiting environment noise; the dynamic noise robustness data processing module is used for carrying out noise reduction processing, semantic understanding and closed-loop optimization on the voice signal and generating feedback information and a control instruction; the AR multi-modal interaction compensation module is used for receiving feedback information of the dynamic noise robustness data processing module and providing visual and tactile interaction compensation, and the job management module is used for scheduling management equipment according to a control instruction; according to the method, the operation efficiency can be effectively improved, the accuracy is high, the real-time performance is high, and interaction experience optimization can be realized.

The invention relates to the technical field of robot tactile perception, in particular to a laparoscopic surgery robot tactile interaction system and method, and the system comprises an F / T sensor which is used for obtaining and detecting sensor information of contact force between a tail end surgical instrument and patient tissue; the data processor is used for detecting the contact force between the tail end surgical instrument and the tissue of the patient according to the information of the sensor, converting the contact force into feedback force which can be sensed by an operation object, converting the feedback force into control parameters of the force feedback device, and smoothly optimizing the control parameters; and the force feedback device is used for controlling and driving a force feedback type main manipulator of the laparoscopic surgery robot according to the control parameters after smooth optimization so as to transmit feedback force to an operation object for sensing. According to the invention, an operator can sense the contact force between the end surgical instrument and the tissue of a patient, the operator can be prevented from sensing abrupt change, the smoothness of tactile sensing is ensured, and a basic guarantee is provided for accurate tactile judgment of the operator.

The invention discloses a cognitive impairment risk intelligent matching intervention system based on multi-modal data, and belongs to the technical field of cognitive impairment, and the cognitive impairment risk intelligent matching intervention system specifically comprises the following steps: collecting a tactile interaction sequence and a face video data stream when a user executes a cognitive intervention task, and constructing an original multi-modal data set; extracting tactile operation track features and reconstructing a heart rate variability sequence, and generating a synchronous multi-mode feature set; identifying a difference interval of state fluctuation and extracting behavior and physiological features in the interval to form an instant physical and mental load parameter set; in combination with an interaction efficiency index in the task execution record, generating a joint state feature vector, and outputting a personalized adaptation instruction through a pre-trained sub-state-parameter association model; and constructing an adaptive strategy model by using the multi-modal features and an adaptive instruction to realize millisecond-level dynamic adjustment from the real-time multi-modal features to task parameters. According to the method, real-time perception and accurate matching of the cognitive emotion load of the user are realized, and the individuation degree and instantaneity of intervention are improved.

The invention relates to a virtual tactile simulation method of a tactile glove, which comprises the following steps of: establishing a perception response pedigree model by analyzing the surface attribute of a virtual object; projecting touch interaction of the user in the virtual scene into a multi-level force feedback target value; sensing spectrum parameterization is introduced, and a feedback sensitive interval is set according to the minimum distinguishable difference JND principle of sensing psychology; dividing the finger tip region into a plurality of sub-sensing regions by adopting a region specific response function modeling mechanism; a flexible film strain sensor is used for detecting the finger pulp contact position and distribution in real time; judging whether the contact is concentrated, diffused or slipped or not, and dynamically adjusting a feedback strategy of a corresponding area; introducing a feed-forward predictive modulation mechanism: predicting a contact state in the future 5-10 ms by using motion capture; generating a response signal in advance, and sending the response signal to a driving layer to offset control-sensing lag; a bistable cooperative driving structure is adopted; a channel distribution mechanism of contact importance is adopted; and adjusting resource occupation according to the interaction task semantics.

The invention provides an intelligent building system interaction experience method and system combined with VR, and the method comprises the steps: collecting a continuous behavior sequence of a user in a physical space through a VR interaction terminal, and generating virtual-real behavior mapping data containing the association relationship between the physical behavior of the user and a virtual building scene function region; based on the virtual and real behavior mapping data, constructing a user intention evolution trajectory, covering an intention change trend of a time dimension and a region attention path of a space dimension, and generating a dynamic reconstruction instruction containing a space layout adjustment rule and an interaction element activation strategy according to the intention evolution trajectory; driving real-time spatial recombination of a virtual building scene according to the dynamic reconstruction instruction, generating a multi-modal feedback signal containing visual scene update data and tactile interaction response data, collecting a real-time response behavior of a user to the feedback signal, optimizing a prediction model of an intention evolution trajectory, and performing real-time reconstruction on the intention evolution trajectory. Therefore, the dynamic adjustment of the virtual building scene according to the intention of the user is realized, and personalized and immersive interaction experience is provided.

The invention relates to a force feedback simulation system and method based on a high refresh rate, the system comprises a tactile interaction device and a data processing feedback module, the tactile interaction device comprises an input signal acquisition unit used for acquiring information of the tactile interaction device and feeding back the information to the data processing feedback module; the feedback output unit is used for receiving the feedback signal and executing a simulation action; the data processing feedback module comprises a stress dynamic model which is used for respectively constructing a plurality of models according to different stages of simulation and determining and outputting a feedback signal after receiving the information of the mobile terminal; and the data acquisition refresh rate of the input signal acquisition unit is greater than or equal to 1000Hz. According to the method, the tactile fidelity, continuity and operation effect of operation simulation are remarkably improved, so that the method is closer to a real operation environment, the skill training and practice level of surgeons is improved, and the method is not only suitable for simple single-layer tissue puncture, but also supports complex multi-layer tissue simulation, and can be widely applied to minimally invasive operation training scenes.

The invention provides an autism child personalized emotion perception method based on tactile interaction, and the method comprises an identification stage: employing a trained autism child personalized emotion identification model to identify tactile behavior data in a man-machine interaction process of an autism child and a human-shaped accompanying robot, and obtaining an emotion classification result; in the model training stage, the personalized emotion recognition model of the autism children comprises a public feature extractor, a domain independent feature extraction and classifier, a domain general feature extraction and classifier and an aggregation prediction classifier. The training process of the personalized emotion recognition model for children with autism comprises the following steps: acquiring tactile behavior data; dividing the tactile behavior data into source domain data and target domain data; obtaining a source domain feature and a target domain feature through a common feature extractor; obtaining a domain independent classification result through a domain independent feature extraction and classifier, and obtaining a domain general classification result through a domain general feature extraction and classifier; and obtaining a classification result of the emotion category through an aggregation prediction classifier.

The invention relates to the technical field of tactile interaction, in particular to a virtual material interaction system with tactile feedback, which comprises a contact state sensing module, a potential gradient calculation module, a friction dynamic regulation and control module, a tactile feedback generation module and an interaction state monitoring module. According to the invention, dynamic mapping of contact pressure, sliding speed and node potential is realized through multi-source signal fusion, so that touch perception keeps continuous response in time sequence, potential difference calculation forms an electric field regulation basis for microscopic change of a material, and space-level adjustment of a surface friction coefficient is realized through charge redistribution and polarization direction switching. The tactile feedback changes adaptively according to material features, the friction spectrum is recombined into a multi-frequency-domain tactile signal after Fourier decomposition, texture, hardness and damping features of different materials are presented in a layered mode, the tactile feedback has more dynamic details and direction consistency, the sense of reality and the tactile recognition degree of virtual materials are improved, and the sense of reality and the sense of touch recognition degree of the virtual materials are improved. And stable and natural interaction touch in the virtual environment is ensured.

The invention belongs to the technical field of tactile communication and intelligent man-machine interaction, and discloses a multi-point tactile intelligent encoding and decoding device, system and method, and the method comprises the steps: obtaining an original tactile signal through an acquisition end, and extracting features through a neural network auto-encoder; a weighted fusion module based on an attention mechanism combines channel state information (CSI) to carry out dynamic grouping residual quantization on coding features, and low-code-rate compression transmission is realized; the sensing end inversely quantizes received data and reconstructs a signal, and outputs a tactile signal in combination with a multi-scale sensing calibration mechanism in an optimized manner, so that the tactile signal is closer to original input in a human body sensing dimension; the multi-point tactile feedback device obtains vibration feedback corresponding to the real-time multi-channel vibration tactile signal based on the reconstructed signal. By fusing the deep learning architecture and the psychological tactile perception model, efficient compression, low-delay transmission and high-quality reconstruction of tactile signals are achieved, and the method is suitable for high-precision tactile interaction scenes such as virtual reality, remote operation and medical robots.

The application discloses a flexible tactile sensing and feedback system and method based on texture recognition, which comprises an electrostatic flexible tactile sensor, a feedback actuator and a tactile feedback control circuit, the electrostatic flexible tactile sensor comprises a tactile interaction film for acquiring texture features and a pressure sensor for texture feature conversion; the feedback actuator completes tactile feedback of the texture feature signal acquired by the flexible sensor in the form of vibration; the tactile control system is mainly connected with the electrostatic flexible tactile sensor and the tactile feedback actuator through the feedback control circuit; the tactile feedback actuator has a stimulation reinforcement structure on the contact surface with the human body, which can further enhance the fine feedback of the tactile feeling. The application can effectively identify the texture features and complete the feedback of the fine tactile feeling in the form of vibration, provides effective assistance for the tactile recovery of the disabled, and has important application value in the game industry, especially in the development of virtual reality technology.

The invention relates to the technical field of virtual reality, in particular to a spherical tactile feedback device, a driving method and a virtual reality system.The spherical tactile feedback device is used for virtual reality operation feedback and comprises a spherical shell, at least one actuator, at least one curved surface sliding unit and a fixing support; the spherical shell comprises a plurality of deformable surfaces for tactile interaction; the curved surface sliding unit is arranged in an accommodating space in the spherical shell along the radial direction and is used for controlling the deformation of the outer surface of the spherical shell; the actuator is used for providing kinetic energy for the moving end of the curved surface sliding unit; the fixing support is located at the sphere center position of the spherical shell, and the fixing support is located at the sphere center position of the spherical shell and used for containing the actuator. Tactile feedback is provided through the volume change of the spherical tactile feedback device, and multi-dimensional dynamic deformation is achieved; and the three-dimensional relation perception experience and interaction experience of the user in the virtual reality space are improved.

This invention discloses a force feedback method, device, electronic device, and storage medium. The method includes: determining a target surgical instrument; during a simulated surgery performed on a skin tissue model using the target surgical instrument, determining a first real-time position of the pierceable tissue entity at a target time point, and determining a first reference position of the moving entity on the tissue at a reference time point corresponding to the target time point; determining a second real-time position of the moving entity on the tissue at the target time point based on the first real-time position information and the first reference position information; determining a target feedback force of the target surgical instrument at the target time point based on the first and second real-time position information; and providing force feedback to the object operating the target surgical instrument based on the target feedback force. Based on this invention, force feedback-based simulated surgery can achieve a tactile interactive perception that more closely resembles actual surgery.

An immersive scene description is defined by a data structure containing information representing triggers and haptic effects. When a trigger occurs while the user is interacting with the immersive scene, parameters for the haptic effect are retrieved from the data structure and provided to the haptic actuator for rendering the haptic effect. These parameters determine at least the modality and perception of the haptic effect, and can determine at least some of the body parts to which the haptic effect should be applied. The modality relates to the type of haptic effect, such as temperature, vibration, pressure, or acceleration. Perception determines the haptic signal to be applied to the corresponding haptic actuator. This signal is generated or retrieved from a file or other device.

The application discloses a non-contact force haptic reproduction system and method based on three-dimensional orthogonal background electromagnet, which realizes force haptic reproduction in a real operation space in a three-dimensional orthogonal background electromagnet through a three-axis fingertip electromagnet worn on a fingertip, wherein the fingertip electromagnet is composed of sub-modules, and the background electromagnet is composed of sub-modules; simulation obtains influence of force of the background electromagnet on the fingertip electromagnet, and obtains a mapping relationship of the force of the fingertip electromagnet to the background electromagnet excitation current in the whole three-dimensional operation space; three-dimensional position coordinates and rotation angles of a human finger are obtained by an optical method, and are sent into a force haptic interaction model to calculate target three-dimensional feedback force; based on offline simulation data and given three-dimensional position coordinates, rotation angles and target three-dimensional feedback force of the human finger, excitation currents of the background electromagnet and the fingertip electromagnet are finally determined.

A system for managing interactive communication includes user devices linked to users, member devices linked to members, an interactive user interface module coupled with the user devices, and a content management module coupled with the member devices. A communication module establishes network communication among the user devices, member devices, interactive user interface module, and content management module. Member devices are configured to pair with user devices. The content management module handles various interactive content types to be displayed by the interactive user interface module on user devices, either automatically or in response to user haptic interactions. Further, a method for managing communication is also provided.

The invention provides an ICU patient remote ward-round system and method based on VR, and the system achieves the immersive remote presentation of the state of an ICU patient through high-precision data synchronization, multi-mode interaction design and a low-delay rendering architecture, remarkably improves the doctor-patient communication efficiency and the psychological pacifying effect of family members, and improves the patient experience. And a brand-new digital solution is provided for an intensive care scene. A doctor can quickly position the abnormal states of multiple patients through a VR scene (for example, the doctor can check the breathing amplitudes of the patients in 10 wards at the same time), and the ward round efficiency is improved by 300% (the traditional polling ward round needs 15 minutes for each patient, and the VR system only needs 3 minutes). Clinical tests show that after VR is used for visiting, the score of an anxiety scale (GAD-7) of family members of a patient is averagely reduced, and the score is mainly attributed to'real contact feeling 'and vital sign transparency caused by tactile interaction.

The invention discloses a rehabilitation robot auxiliary control system and method based on multi-modal perception, and belongs to the field of robot auxiliary rehabilitation and mobile support. The method comprises the following steps: acquiring and processing tactile interaction information between a user and a robot, and calculating contact force, torque and a contact position; in order to ensure stable and reliable support, centroid dynamics is used for modeling a human body state, and nonlinear model predictive control is used for optimizing a sequence of contact force and a contact position; inputting the contact force, the torque and the contact position into an admittance controller, generating a track adjustment amount, and performing adaptive correction on the reference track; solving a joint angular velocity control instruction of the robot by adopting a quadratic programming method; the joint position, the speed limit and the self-collision avoidance constraint are strictly included in the quadratic programming problem, and the safety of the whole interaction process is ensured. The walking ability can be remarkably improved, emergency situations such as falling down can be effectively handled, and the walking aid is particularly suitable for the field of elderly nursing.

A haptic interaction information processing method, device, and apparatus are disclosed, which belong to the field of haptic signal representation. The haptic interaction information processing method according to an embodiment of the present disclosure includes: counting the number of waveforms of haptic interaction signals in a target video screen, where the target video screen is one frame video screen in the target video, and one waveform of haptic interaction signal corresponds to the haptic interaction information of one video screen area in the target video screen (S101); and generating a grayscale information representation diagram corresponding to the target video screen based on the number of waveforms of the haptic interaction signals, where the grayscale information representation diagram is for representing the distribution position of each video screen area in the target video screen (S102).

The invention relates to the technical field of medical rehabilitation equipment control, in particular to a leg fracture bedridden complete-cycle rehabilitation system based on hybrid real force touch interaction and a control method, by adopting the method provided by the invention, a multi-mode rehabilitation state feature set is constructed, a coupled feature set is formed through feature coupling, and the rehabilitation effect of a patient is improved. The method comprises the following steps: firstly, dynamically updating a local dynamic transition threshold value and a global dynamic transition threshold value, setting a trigger condition, judging whether to transition from a current rehabilitation stage to a next stage or not according to the trigger condition, and dynamically updating the local dynamic transition threshold value and a global accumulation threshold value; and a control signal for virtual display is sent to the MR interaction unit according to the current rehabilitation stage and a preset condition, so that real-time adaptive switching of multi-stage rehabilitation training is realized, a mixed reality function is realized in combination with the MR interaction unit, lightweight, immersive and personalized rehabilitation experience is provided for a patient, leg function recovery is accelerated, and the rehabilitation effect of the patient is improved. The rehabilitation efficiency and quality are improved.

The application discloses a feedback method of a social robot based on tactile interaction and the social robot, and the feedback method comprises the following steps: a sensing module acquires petting data of a visitor and sends the petting data to a control chip, a tactile feedback module and a motion simulation module; the tactile feedback module provides tactile feedback to the visitor according to the petting data, and the motion simulation module performs initial simulation motion according to the petting data; the control chip classifies the petting data to obtain current petting classification and sends the current petting classification to the tactile feedback module and the motion simulation module; and the motion simulation module adjusts the initial simulation motion according to the current petting classification to provide visual feedback to the visitor. The application does not need to occupy the attention of the visitor and can reduce the consultation pressure of the first-time visitor.

A touch screen device, a touch screen stylus, and a method of providing haptic feedback to a user of a touch screen device. The method includes the steps of: identifying a relative position of a tip of the touch screen stylus with respect to a sensing surface of the touch screen device; asserting a haptic force signal that changes in response to the relative position; and passing a controlled variable current through an electromagnetic circuit to apply a magnetic force between the touch screen device and the touch screen stylus, wherein the current changes depending on the haptic force signal.

The application provides a natural tactile interaction method and component of a data-driven bionic robot, and the method comprises the following steps: collecting tactile interaction process data of a real organism corresponding to the bionic robot and a user and statistically aggregating the data to obtain a training data sequence; establishing a data-driven interaction prediction model and training the data-driven interaction prediction model by using the training data sequence; determining a tactile input signal generated by a tactile input system of the bionic robot acted on by the user; and controlling the natural tactile interaction of the bionic robot according to the tactile input signal by using the trained data-driven interaction prediction model. By the method of the application, the touch action of the user can be recognized by the tactile input system, the natural feedback that the bionic robot should give to the user can be calculated by the data-driven interaction prediction model, and the closed loop of the natural tactile interaction feedback can be realized by controlling the action of the bionic robot.

The application relates to the technical field of virtual reality, in particular to a VR tactile feedback system based on multi-modal perception fusion and a method thereof, which comprises a tactile interaction terminal, a distributed tactile feedback terminal, a distributed signal acquisition unit, a computer system, a tactile presentation device, a visual device and an auditory device; the core of the application is a tactile prediction unit in the distributed tactile feedback terminal; a 3D content tactile prediction algorithm based on matrix theory is constructed; tactile texture matrix and tactile material matrix are used to realize accurate mapping of visual features and tactile features; a high-dimensional matrix is innovatively used to represent a tactile feature space; a matrix spectrum analysis technology is used to realize adaptive adjustment of tactile features; a multi-scale matrix decomposition and fusion algorithm is used to realize tactile texture reconstruction from a macroscopic view to a microscopic view; the application solves the problems of limited tactile feature expression capability, lack of adaptability and insufficient multi-modal perception cooperation in a traditional VR tactile feedback system.

The invention relates to the technical field of psychological intervention and tactile interaction, in particular to a group psychological sand table somatosensory interaction device based on tactile interaction, which comprises a pressure dynamic fingerprint acquisition unit and a dynamic adjustment feedback unit. A pressure dynamic fingerprint containing a pressure topological structure variation index and a pressure gravity center migration acceleration is obtained, a dynamic adjustment feedback unit inputs the pressure dynamic fingerprint into a lightweight space-time diagram convolutional network optimized by adversarial training, synchronously outputs an operator identity label and an operation intensity level, activates a corresponding wrist strap tactile feedback device, dynamically adjusts parameters, and outputs a corresponding wrist strap tactile feedback result. When the intensity is low, high-frequency micro-vibration is triggered, when the intensity is high, progressive pressure waves are triggered, when the similarity of two sand tool pressure fingerprints exceeds a threshold value, freezing is carried out, then a sand tool feedback channel is operated, and the equipment improves feedback accuracy and interaction orderliness and is suitable for group psychological sand table treatment.

The present application relates to the technical field of wearable terminal surface microstructure and tactile interaction, and discloses a wearable terminal surface structure forming method and a wearable terminal, the method comprising: obtaining target surface information of the wearable terminal, the target surface comprising at least one of an interaction area and a wearing contact area; establishing a microtexture database, storing texture parameters, friction characteristics and tactile distinguishability indexes of candidate microtexture structures under dry, aqueous wetting and oily wetting working conditions; dividing functional partitions according to the functional attributes of the target surface, and determining corresponding microtexture structures for each functional partition based on the microtexture database, so that the target surface forms a tactile resistance distribution and / or a friction regulation distribution; the microtexture database and texture layout rules are updated through prototype verification data backflow, so that the tactile resistance gradient corresponds to the continuous parameter risk interval, and the synergistic optimization of wearing stability, comfort and continuous parameter adjustment safety prompt is realized.

The invention belongs to the technical field of man-machine interaction and force sense feedback, and particularly relates to a force sense control method for a virtual orthopedic surgery system, which comprises the following steps of: acquiring a displacement signal generated when an operator operates a force feedback master hand; tool stress in the virtual environment is calculated through a virtual model based on the displacement signal; acquiring an actual stress signal fed back by the force feedback master manipulator; calculating a force control error between the tool stress and the actual stress; based on the error, a sliding mode controller is adopted to generate a main control quantity; meanwhile, a disturbance observer is adopted to observe the lumped disturbance in the force feedback master hand system in real time, and a disturbance estimation value is obtained; performing low-pass filtering processing on the estimated value to generate an interference compensation amount; and finally, adding the main control quantity and the interference compensation quantity, synthesizing a final control quantity, and outputting the final control quantity to the force feedback master manipulator. Through combination of sliding mode control and observer-based feed-forward compensation, the authenticity and stability of haptic interaction of the virtual operation are remarkably improved.

The invention provides a student practice effect feedback method and system based on multi-dimensional data driving, and relates to the technical field of training feedback systems. According to the method, the expert standard action library is constructed, the joint point activity frequency is dynamically analyzed based on project types, core and secondary capture points are intelligently divided, and the collected data are subjected to grading processing in combination with a weighting mechanism, so that the system efficiency is improved while the key action information precision is guaranteed; a difference value is calculated by performing weighted comparison on an action track of a student and a standard track, layered feedback is generated according to a preset threshold value, differentiated tactile prompts are output through wearable equipment, and closed-loop control from data acquisition and intelligent evaluation to real-time feedback is realized; according to the whole scheme, the accuracy and scientificity of action evaluation are improved, students are stimulated and helped to quickly correct errors and consolidate correct actions in a non-intrusive real-time tactile interaction mode, and the training efficiency, concentration degree and autonomy are remarkably improved.

The invention discloses a volume three-dimensional display system and method based on sound field assembly and photoluminescence nanoparticles. The system comprises a sound field assembly module, an optical excitation module, an aerosol management module, a sensing and interaction module and a central control core. The method comprises the following steps: firstly, generating a programmable sound field in a free space by using an ultrasonic phased array, and accurately assembling nanoparticles into an intangible three-dimensional voxel matrix; and then, the nanoparticles in the matrix are directly excited through multi-band laser scanning, so that the nanoparticles serve as self-luminous voxels to emit light, and a full-color volumetric three-dimensional image is actively rendered. According to the invention, a traditional reflective projection screen is completely abandoned, true three-dimensional and omnidirectional visible display is realized, accurate force touch interaction can be realized through sound field distortion monitoring, and the key bottleneck of the existing aerial imaging technology is solved.

The invention relates to the technical field of man-machine interaction and intelligent control, in particular to a flexible somatosensory interaction system and method based on intention probability mapping. Through setting a flexible somatosensory acquisition module, an intention modeling module, a flexible-rigid mapping field construction module, a dynamic intention screening module and an execution control module, multi-modal somatosensory information of an operator is acquired, a probability distribution model of an action intention is established, and a flexible-rigid mapping field is constructed based on an information retention and minimum semantic loss principle. And the continuous action space of the human body is converted into a limited control instruction set which can be executed by the mechanical arm. The system dynamically screens an optimal control instruction by using a confidence coefficient driving mechanism under multiple candidate intentions, and realizes accurate response of a mechanical arm to the action of an operator through closed-loop feedback. The problems that in the prior art, flexible action and rigid control freedom degrees are not matched, and intention recognition is fuzzy are effectively solved.