1354 results about "Tactile sense" patented technology

The invention relates to the technical field of robot decision making, and provides a robot multi-modal fusion autonomous decision making method and system based on a large language model.The method comprises the steps that a robot obtains multi-modal environment information through a visual sensor, a touch sensor, an auditory sensor and a laser radar which are carried by the robot; performing preliminary filtering and noise reduction processing on the original sensor data, and synchronously recording all the sensor data by timestamps; performing space-time semantic alignment on the preprocessed multi-modal data, mapping pixel coordinates of a target in a visual target coordinate quantization original image to a robot coordinate system, performing uncertainty evaluation on a multi-modal signal through a dynamic Bayesian network, and taking entropy or variance as an uncertainty quantitative evaluation index. According to the method, the information quality is improved from a data fusion source, accurate and reliable basic support is provided for subsequent decision making, and decision making errors caused by data deviation are greatly reduced.

The invention relates to the technical field of artificial intelligence, can be applied to service scenes such as pension service, financial science and technology and medical health, and discloses a grabbing control method, device and equipment based on collaborative grabbing characteristics and a medium. Generating collaborative grabbing features and feature weight vectors, constructing a grabbing model in combination with historical grabbing data, and determining an initial grabbing strategy; and in combination with an initial grabbing strategy and real-time tactile feedback data, a grabbing control sequence and a cooperative adaptation coefficient are generated through constraint optimization, grabbing operation is executed, real-time feedback is monitored, the grabbing force and posture are dynamically adjusted according to deviation and the cooperative adaptation coefficient, and accurate control is achieved. According to the method, the problem that in the prior art, grabbing response is not flexible is solved by constructing the collaborative grabbing features and the grabbing model and fusing real-time feedback and control strategies, dynamic self-adaption and fine adjustment in the grabbing process of different objects are achieved, and the damage risk is reduced.

The invention relates to a robot dynamic grabbing control method based on visual sense and tactile sense depth fusion. The method comprises the following steps that firstly, based on visual information, surface geometric features of a target object are extracted, grabbing adaptability scores are calculated, an optimal grabbing point is selected, and a collision-free approaching track is planned; secondly, in a contact establishment stage, detecting a contact event through a touch sensor, fusing vision and touch data to unify a coordinate system, calculating a vision-touch consistency score, and applying an initial grabbing force; and thirdly, in the stable holding stage, slip detection is conducted through wavelet packet energy entropy and pressure gradient, and the grabbing force and impedance parameters are dynamically adjusted in combination with self-adaptive impedance control. According to the method, intelligent control over the whole process from approaching to contact to stable holding is achieved, and the adaptability, stability and safety of robot grabbing in the dynamic environment are improved.

The invention relates to the technical field of electromyographic signal monitoring, and discloses a guided monitoring method and system based on a wireless surface electromyographic analysis feedback instrument. The method comprises the following steps: acquiring electromyographic signal time sequence data and limb movement track space coordinate information when a target muscle group moves through a multi-channel wireless surface electromyographic sensor; muscle activation intensity and fatigue change features are extracted, a joint activity angle change sequence is calculated, and a muscle-joint collaborative activity feature matrix is generated through space-time alignment fusion; analyzing signal transduction delay and amplitude matching degree in the matrix, and evaluating the nerve driving coordination level; according to the deviation from the preset standard mode, generating a real-time biological feedback signal containing an action correction instruction and a muscle activation intensity adjustment parameter; and converting into a tactile vibration mode and a visual guidance mark to be output. According to the method, synchronous acquisition and fusion analysis of multi-dimensional motion information are realized, and visual guidance is provided through multi-modal feedback.

The invention discloses a touch feedback enhanced multi-mode robot grabbing control system and method. The touch feedback enhanced multi-mode robot grabbing control system comprises a touch feedback enhancement module, a visual touch language large model semantic generation module and a touch enhancement diffusion control optimization module. The tactile feedback enhancement module is based on a multi-agent cooperation system, performs semantic level and signal level processing on tactile signals, and realizes upper and lower layer coordination through a feedback routing mechanism. And the visual touch language large model module adopts a touch encoder based on CLIP to realize multi-mode semantic conversion and material attribute identification. And the tactile enhancement diffusion control module adopts a BRIDGeR style interpolation diffusion technology, and combines visual embedding and tactile constraint to realize high-frequency optimization of an action track. According to the method, the material identification accuracy, the precision operation success rate and the complex scene self-adaptive capability are remarkably improved.

A robot control method based on tactile prediction pre-training comprises the following steps: acquiring and generating a human playing data set consisting of three-channel image tensors in an offline stage, and training a constructed conditional diffusion model comprising a tactile encoder, a tactile decoder and an action and visual encoder; in the online stage, the trained conditional diffusion model is integrated into a standard imitation learning strategy network, and an action instruction of the robot is generated according to the state of the robot, the current visual features and the tactile feature vectors extracted by the imitation learning strategy network. According to the method, a specific agent task is completed by training a deep neural network model, that is, a future tactile signal sequence is predicted according to historical information and future action intentions; the model is enabled to characterize generic haptic features contacting physical dynamic laws for further migration into downstream robot control tasks.

The invention relates to the technical field of robot control, and discloses an industrial robot adaptive control method and system based on multi-modal sensor fusion, and the method comprises the steps: collecting multi-modal original data, and carrying out the time-space alignment; capturing space-time semantic association of visual textures, tactile pressure distribution and force sense fluctuation in the multi-modal data through a multi-head attention mechanism guided by a physical model, and performing space-time registration; a CNN-LSTM hybrid model is adopted to extract visual texture features and time sequence tactile features in the physical information enhanced multi-modal feature matrix; and carrying out dynamic weight distribution on the fusion feature vectors with physical consistency by utilizing a weight distribution model driven by element reinforcement learning to generate dynamic weighted fusion features. According to the method, the spatial positioning precision of the industrial robot in a precise assembly scene is greatly improved, the contact force control stability is greatly improved, and the control robustness in a complex operation scene is remarkably enhanced.

The invention relates to the technical field of blind guiding robot interactive navigation, and particularly discloses a blind guiding robot interactive navigation system combining vision, inertial navigation and voice, which is characterized in that environmental vision, movement and voice data are synchronously acquired through a multi-modal perception data acquisition module, and are analyzed through a situation-user joint state feature extraction module; a user potential intention deduction module is combined to predict a user intention and generate a self-adaptive guide strategy; a self-adaptive guiding path optimization module adjusts a navigation path according to a strategy to ensure safety and high efficiency; finally, the situational multi-mode interaction instruction output module provides voice and tactile feedback, and interaction experience and navigation autonomy are enhanced. According to the method, the travel problem of the visually impaired person in a complex environment is solved, the individuation, context perception capability and safety of blind guiding are improved, and more natural and efficient navigation assistance is brought to the visually impaired person.

The invention relates to the field of robot intelligent control and man-machine collaboration, in particular to a multi-sensing collaborative robot control method, which comprises the following steps: synchronously filtering mechanical, tactile, visual and physiological signals to generate a unified observation frame, and calculating power flow and energy in combination with a joint state to form an execution vector; extracting a topological descriptor by using persistent coherence, and outputting an interaction potential field and a human body intention by using the topological descriptor as a conditional driving diffusion model; then, power density is constructed based on the potential field gradient and intention, joint torque and impedance are generated through entropy regular optimal transmission and Jacobian and microoperator kernel mapping, and an energy Lagrange multiplier is updated in real time; and if the consistency error continuously exceeds a threshold value, estimating a Koopman lifting model on line by using a causal weight, completing model prediction control under the energy, conformal security and topology consistency constraint, outputting an update instruction and performing closed-loop feedback. The method improves the cooperation precision and safety, reduces the energy consumption, and is suitable for a high-dynamic man-machine cooperation scene.

The invention relates to the technical field of vehicles, in particular to an intelligent cabin man-machine cooperative control method and device, a vehicle, a medium and a product. Performing fusion processing on the voice signal data, the visual signal data and the tactile signal data of the current driver by using a preset dynamic weight distribution model to obtain a multi-modal fusion control instruction; and under the condition that the multi-mode fusion control instruction meets a preset sight-voice dual-mode condition, synchronously issuing the multi-mode fusion control instruction to a plurality of target execution devices in the intelligent cabin so as to drive each target execution device to perform cooperative control. Therefore, by constructing a cooperative control mechanism of multi-mode sensing and dynamic weight distribution, the problems that the false wake-up rate is high, the voice and graphical interface cooperative efficiency is low and the interaction mode cannot adapt to a complex driving environment due to a single or separated interaction mode in the related technology are effectively solved.

The invention discloses an operation AR intelligent auxiliary management system based on voice AI interaction driving, which is applied to the field of intelligent warehousing and comprises an environment perception type voice interaction module, an AR multi-mode interaction compensation module, a dynamic noise robustness data processing module, an operation management module and a database module. The environment perception type voice interaction module is used for collecting voice signals of operating personnel and inhibiting environment noise; the dynamic noise robustness data processing module is used for carrying out noise reduction processing, semantic understanding and closed-loop optimization on the voice signal and generating feedback information and a control instruction; the AR multi-modal interaction compensation module is used for receiving feedback information of the dynamic noise robustness data processing module and providing visual and tactile interaction compensation, and the job management module is used for scheduling management equipment according to a control instruction; according to the method, the operation efficiency can be effectively improved, the accuracy is high, the real-time performance is high, and interaction experience optimization can be realized.

The invention relates to the technical field of mechanical arms, in particular to a chemical emergency disposal-oriented manipulator cross-modal sensing control system which comprises an environment scanning module, a visual positioning module, an obstacle avoidance planning module, a pose compensation module, a decision mapping module and a grabbing updating module. Obtaining environment state information; an operation view angle image of the manipulator is collected, visual features of the operation view angle image are solved, and pose information is obtained; obstacle avoidance correction is conducted on the motion trail of the mechanical arm, and a collision-free approaching path is obtained; the operation view angle image and the multi-mode micro-touch signal are fused, error compensation is carried out on the pose information, and an accurate grabbing pose is obtained; mapping object surface physical attributes sensed by the multi-mode micro-tactile signals to obtain a self-adaptive grabbing instruction; the manipulator is controlled to conduct grabbing through the self-adaptive grabbing instruction, and environment state information is updated; according to the invention, the accuracy of cross-modal sensing control of the manipulator can be improved.

The invention relates to the technical field of data processing, in particular to a method for driving emotion interaction of an intelligent device based on multi-modal understanding, and aims to eliminate illumination and noise interference and output a standardized face video stream, an effective voice segment and a touch thermodynamic diagram through an environment adaptive acquisition module. The feature extraction module extracts facial action optical flow features, voice Mel-frequency cepstral coefficient vectors and tactile pressure gradient parameters. The cross-modal correlation model adopts a tensor decomposition algorithm to calculate a space-time correlation matrix of visual and voice features, and the tactile feature weight is dynamically adjusted in combination with environmental parameters. According to the response strategy, an intervention scheme is retrieved based on a graph database, emotion confirmation statements, guide statements and behavior suggestions are fused to generate multi-mode response, and PID adjustment of the temperature control device and tactile pulse output of the vibration device are synchronously driven. And the feedback evaluation module verifies the emotion recognition consistency through a Pearson's correlation coefficient, triggers conflict sample separation storage and model increment training, and realizes closed-loop optimization.

The invention relates to the technical field of artificial intelligence, and discloses an artificial limb driving method based on a spiking neural network, which comprises the following steps: acquiring a multi-modal signal, and preprocessing the multi-modal signal; converting the preprocessed multi-mode signal into a pulse signal; inputting the pulse signal into a pulse neural network for sensing fusion processing, and outputting a control strategy; dynamically adjusting a control strategy based on pulse reinforcement learning according to the behavior feedback information through a reward function, and generating a learning result; and generating a bionic motion instruction according to the adjusted control strategy and the learning result, and driving an artificial limb execution mechanism to act according to the bionic motion instruction. According to the method, the electromyographic signals, the tactile pressure signals and the inertial measurement data are uniformly coded into the pulse sequence, the pulse neural network is utilized to realize multi-modal sensing fusion, the pulse reinforcement learning algorithm is introduced, the artificial limb control strategy is optimized in real time according to user behavior feedback, and the accurate control of the artificial limb joint is improved.

The invention provides a multi-modal fusion fatigue screen monitoring recognition and reminding method and system, and the method comprises the steps: data collection: collecting the visual data of a screen monitoring person in real time, and synchronously collecting the physiological data; multi-modal data fusion: adopting a feature weighted fusion method based on an entropy weight method to adaptively distribute weights and generate a fusion fatigue index FFI by quantifying dynamic information entropy of each visual data and physiological data; fatigue grade classification: realizing three-level fatigue state judgment based on a fusion fatigue index FFI obtained by multi-modal data fusion and a dynamic decision tree model; and dynamic intervention: based on a fatigue grade classification result, adopting a double-channel intervention mechanism of bracelet touch alarm and automatic telephone call value length. When fatigue and inattention of a monitoring screen watchman occur, the monitoring screen watchman can be timely and accurately identified and a reminding intervention mechanism is started, so that the continuity and the safety of the monitoring screen work are ensured, and bad safety production events caused by human reasons are avoided.

The invention discloses a robot system with tactile perception, and belongs to the technical field of tactile robots. The problem that an existing robot is poor in environment pressure sensing capacity is solved. Comprising a head module, an integrated vision-inertial navigation sensing module and a head controller, two side surfaces are provided with a pair of rotatable connecting ends, and the outer surface is provided with a tactile sensor; the head controller transmits data to the upper computer through the communication module; the driving body module comprises a plurality of modularized joints which are connected end to end, touch sensors are arranged on the two side faces of each modularized joint, and the adjacent modularized joints are orthogonally connected; the modularized joint is integrated with a joint controller and a joint steering engine; the tail module integrates a tail controller and a tail steering engine, and tactile sensors are arranged on the two side faces of the tail module. The tail controller carries out data transmission with the head controller through the adjacent previous joint controller in sequence, and the head controller receives a control instruction of the upper computer and carries out data transmission with all other controllers. According to the invention, the environmental pressure can be sensed in all directions.

The invention provides an intelligent tactile stick interaction method and system based on multi-modal perception and edge calculation, and relates to the technical field of intelligent auxiliary equipment. The method comprises the following steps: acquiring environment and position data through a camera, a GPS and an inertial measurement unit; performing multi-modal data fusion and deep learning identification by using an edge calculation module to generate a dynamic environment map; performing real-time path planning and obstacle avoidance based on a map and a destination; analyzing a user instruction through voice recognition and interactively adjusting a path; haptic and auditory bimodal feedback is provided through a vibration module and a voice synthesis module according to a navigation result; and when an emergency obstacle is detected or a user actively alarms, a local and remote alarm mechanism is triggered. The system correspondingly comprises an intelligent tactile stick terminal and a cloud server. According to the invention, accurate and real-time perception and intelligent navigation of the environment are realized, visual and natural interaction experience is provided, an effective emergency help channel is established, and the safety, independence and convenience of travel of visually impaired people are significantly improved.

The invention discloses a green plum fruit picking robot based on multi-source data fusion and a picking method thereof, and relates to the technical field of agricultural intelligent equipment. The robot comprises a crawler-type self-adaptive chassis, a multi-degree-of-freedom mechanical arm, a multi-source sensing module, a bionic flexible end effector, an intelligent control unit and a fruit collecting box, and all the parts are electrically connected with the intelligent control unit. The multi-source sensing module synchronously collects vision, spectrum, touch, acoustics and environment data, and accurate recognition and positioning of mature fruits are achieved through hierarchical weighted attention fusion algorithm processing. And fruit damage is reduced in combination with a path planning algorithm introducing a flexible obstacle avoidance coefficient and a flexible picking execution mechanism. The self-adaptive chassis can adapt to complex terrains, and parameters are dynamically optimized through reinforcement learning in the picking process. The green plum picking machine solves the problems that traditional picking is low in efficiency and high in damage rate, and existing equipment is poor in adaptability, and the intelligent level and operation quality of green plum picking are improved.

The invention discloses a DIKWP-driven individualized brain-map interactive feedback system, which is used for neural rehabilitation and brain-computer interface training. The system obtains brain activity data of a patient through electroencephalogram acquisition equipment, constructs a personal brain-semantic map in combination with cognitive evaluation, and establishes a mapping relation between semantic units and brain region responses. In the training process, the DIKWP semantic analysis module performs multi-layer analysis on indexes such as reaction time, accuracy and intention achievement, generates multi-mode instant feedback such as visual sense, auditory sense or tactile sense, and performs directional reinforcement on a weak semantic domain. The system has a dynamic target optimization capability, and the difficulty can be automatically adjusted according to training performance; when attention distraction, emotion abnormity or semantic deviation is detected, a safety intervention mechanism is automatically triggered, and training effectiveness and safety are guaranteed. According to the method, closed-loop individualized rehabilitation interaction is realized, the adaptation degree and efficiency of brain-computer interface training are remarkably improved, and the method is suitable for various rehabilitation scenes such as languages, movement and cognition and has a good industrial application prospect.

The invention discloses a multi-modal emotion calculation method and system, and mainly relates to the technical field of artificial intelligence and human-computer interaction. Comprising the following steps: synchronously collecting voice, visual and tactile multi-modal data of a user, and carrying out feature extraction on each modal data; performing cross-modal fusion on the extracted multi-modal features, including time sequence alignment and semantic association modeling, and generating a global emotion feature vector; performing dynamic emotion reasoning based on the global emotion feature vector, and outputting an emotion category and emotion intensity; generating a tactile feedback signal according to the emotion category and the emotion intensity, and driving an actuator to output; and updating the emotion memory graph based on user interaction data to complete personalized model self-adaption. The method has the beneficial effects that high-precision and low-delay emotion recognition and natural tactile feedback in a cross-culture scene are realized, and the core bottleneck of dynamic drift adaptation failure and emotion-behavior feedback decoupling in the prior art is solved.

The invention relates to a flexible dual-mode capacitive touch sensor based on a three-gradient micro-dome porous structure. The sensor adopts a sandwich structure and comprises an upper electrode layer, a TGP dielectric layer and a lower electrode layer, wherein the upper electrode layer and the lower electrode layer respectively cover the upper surface and the lower surface of the TGP dielectric layer; the TGP dielectric layer comprises a gradient part and a porous part, the porous part is located on the upper surface of the lower electrode layer, and the gradient part is distributed on the porous part; the gradient part comprises three kinds of micro-dome structures and flat-top protrusions. Different pressure working characteristics of the two structures are fully utilized, so that the porous structure works in a low-pressure area, the gradient structure works in a medium-high pressure area, the performance of the porous structure and the gradient structure is synergistically enhanced, the linear range of the sensor is widened, and the sensitivity is further improved.

A robot-based assembly control method includes providing a plurality of neural network models comprising at least a reinforcement learning network model and a tensor fusion network model. Training data for the models includes visual data from the vision device, tactile data from the tactile sensor, motion feedback data from the robot and torque feedback data from the robot. An assembly control system based on the robot, which includes a clamping effector at an end of a movable part of the robot, wherein a tactile sensor and a soft rubber pad are provided on an inside clamping portion of the clamping effector from inside to outside is described. Based on fusing the visual information with the external forces of other dimensions indirectly obtained from changes in tactile signals, representation vectors can be used to generate appropriate instructions of robot motion for flexibly adjusting the insertion force to complete the assembly.

The invention discloses a robot multi-finger grabbing generation method which comprises the steps that a training data set is obtained, the data set comprises target object point cloud for generating candidate poses and tactile priori information used for optimal grabbing pose judgment, and a feature extraction module fusing a dynamic graph convolutional network DGCNN and a multi-scale weighted expansion converter MWDT is used for extracting target object point cloud and tactile priori information used for optimal grabbing pose judgment; extracting geometric features of the point cloud data of the target object; on the basis of the extracted geometric features, a plurality of candidate grabbing poses are generated through a grabbing area generation module and a grabbing pose generation module; and the multiple candidate grabbing poses are input into a grabbing pose optimization module, and the grabbing pose optimization module analyzes the topological structure of the tactile map through a multi-scale map neural network MS-GCN, conducts stability judgment on all the candidate poses and outputs the optimal grabbing pose. According to the method, visual geometrical information and tactile physical characteristics are fused, the limitation that an existing method only depends on the shape of an object and neglects factors such as materials is solved, an accurate and stable grabbing strategy suitable for different objects can be generated, and the grabbing success rate and generalization ability of the multi-finger hand of the robot are remarkably improved.

The embodiment of the invention provides a touch sensing system and method, electronic equipment, a storage medium and a program product. The touch sensing system comprises a contact assembly, a projection assembly, a collection assembly and a processing assembly, and is configured to be capable of being in contact with an external object and generating deformation; the projection assembly is configured to project an active light field with a time modulation characteristic to a first surface of the contact assembly, the active light field is projected on the first surface to form a plurality of pattern areas, the light intensity of adjacent pattern areas changes based on different time sequence codes, and the first surface is opposite to a second surface, making contact with an external object, of the contact assembly; the acquisition assembly comprises a plurality of acquisition units arranged in an array, and is configured to acquire a plurality of pattern areas and output a time stamp acquisition data sequence.

The invention relates to the technical field of attention deficit hyperactivity disorder detection, and discloses a device for detecting cognitive and developmental conditions of children with hyperactivity disorder, which comprises a handle, a head-mounted device, a bracelet, a hierarchical adaptive algorithm module, a multi-dimensional execution module and a data fusion and analysis module, a multi-mode sensing module is arranged in each of the handle, the head-mounted device and the bracelet; a motion capture unit, a high-sensitivity key array, a tactile feedback module and a wireless transmission module are integrated in the handle, and a wireless electroencephalogram acquisition module, a miniature near infrared spectrum module and an eye movement tracking module are integrated in the head-mounted device. According to the invention, through deep fusion of a multi-dimensional cognitive evaluation system and gamepad hardware, the device not only breaks through the limitation of the traditional neuropsychological test in methodology, but also realizes refined capture of ADHD heterogeneity characteristics in the technical level, and provides a hardware basis for establishing an objective and quantitative diagnostic tool with high ecological efficiency.

The invention relates to the field of robot intelligent control, and discloses a robot operation method, system and terminal based on a visual language model, and the method comprises the steps: obtaining a visual image and a natural language instruction, carrying out the combined analysis of the visual image and the natural language instruction through a visual language large model, and generating an initial planning strategy; the robot is controlled to execute operation actions according to the initial planning strategy, and original tactile signals in the operation process are collected; inputting the original tactile signal into a tactile language translation model for feature extraction and semantic mapping to obtain tactile semantic description; when the prompt operation of the tactile semantic description is abnormal or the physical attribute does not accord with the visual expectation, the tactile semantic description is fed back to the visual language large model as an enhanced prompt word; and according to the visual image and the enhanced prompt word, the current task scene is reasoned again, a corrected planning strategy is generated, and the robot is controlled to execute an operation action. The operation precision of the robot is improved.

Disclosed are reducer instrument locking mechanisms. They can be located near a center pivot joint of the reducer, thereby eliminating a ratchet at the reducer proximal end. The locking mechanisms can include a pawl or latch that travels with one handle from an open position to a closed position. The pawl can fall into a groove formed in an opposing handle such that handles of the reducer can be locked relative to one another, at least with regard to movement of the handles away from one another. When falling into the groove, the pawl can create an auditory and / or tactile indication that a sufficient amount of rod reduction has been achieved to allow for set screw insertion. After set screw insertion, the reducer locking mechanisms can be released to clear it of the groove and allow the reducer to return to its open position.

The invention discloses a dynamic assembly line object grabbing method and system based on multi-modal information fusion, and belongs to the technical field of industrial robot grabbing control, and the method comprises the steps: synchronously collecting the visual information of a dynamic assembly line, the joint motion information of a robot and the touch information of a dexterous finger end, and obtaining a standardized multi-source information data set; constructing a coarse grabbing action generation sub-model and a fine grabbing action generation sub-model; the coarse grabbing action generation sub-model obtains a coarse grabbing action sequence through a de-noising diffusion implicit model; according to the coarse grabbing action sequence and the touch information, a fine grabbing action sequence is obtained through a fine grabbing action generation sub-model; adopting a self-supervised learning mode to train the two models in stages; and based on the trained model, a final fine grabbing action sequence is generated, and the robot is driven to execute dynamic grabbing. The core problems that in the prior art, the approaching precision is insufficient, the grabbing stability is poor, the model adaptability is weak, and a control closed loop is lacked are effectively solved.

The invention discloses a virtual simulation teaching method, device and equipment for cattle pregnancy diagnosis, and the method comprises the steps: constructing a three-dimensional geometric model of a cattle pelvic cavity anatomical structure, obtaining a cattle pelvic cavity model, configuring corresponding physical attribute parameters, and generating a virtual anatomical environment; receiving palpation operation of a user through force feedback equipment, driving a virtual fingertip to move in a virtual anatomy environment, and collecting user interaction data in real time; based on the motion, the operation pressure and the physical attribute parameters of the virtual fingertip, calculating an interaction force of an anatomical structure on the virtual fingertip through soft tissue physical simulation, generating a tactile feedback signal according to the interaction force, and outputting the tactile feedback signal to a user through force feedback equipment; comparing the operation track with an expert standard operation model, and guiding an operation path of the user in real time through force feedback equipment according to a comparison result; and performing skill assessment according to the diagnosis conclusion submitted by the user after completing the palpation operation and the user interaction data, and generating a diagnosis practical training report.

Apparatuses, systems, and techniques to perform a grasp of on object using an articulated robotic hand equipped with one or more tactile sensors. In at least one embodiment, a machine-learned model trained in simulation to grasp a cuboid using signals received from tactile sensors is applied to grasping objects of various shapes in a real-world environment.