144 results about "Hand motion" patented technology

The invention provides a self-supervised continuous 3D hand posture tracking method based on a lightweight inertial measurement unit, and the method specifically comprises the following steps: firstly, obtaining a hand motion signal by using a lightweight consumer-level IMU, and extracting features by using a multi-stage neural network containing a bidirectional long-short term memory (Bi-LSTM) module; the global displacement and orientation of the hand are processed through a wrist posture estimation module, and three-dimensional coordinates of hand skeleton points are solved and calculated by means of a finger motion chain; secondly, a self-supervised completion network is realized by combining a random mask and dual loss, and the features are processed to complete complete attitude reconstruction; and finally, combining network output with kinematics prior, generating a three-dimensional grid with reasonable anatomy, solving the problem that joint deformation is inconsistent with torque, and realizing accurate acquisition of a three-dimensional hand posture. For a dynamic hand motion tracking task, the method can adapt to a sparse IMU deployment scene and realize continuous high-precision tracking; the real-time performance demand of daily interaction can be met, and a low-cost solution can be provided for the fields such as medical rehabilitation and virtual interaction which have strict requirements on attitude precision.

The invention discloses a generative interaction method and device for medium-free aerial imaging, and relates to the technical field of man-machine interaction for medium-free aerial imaging. The method comprises the steps of obtaining a reference description text, hand motion data and hand posture data corresponding to a target user, wherein the hand posture data is used for indicating a final posture of hand motion; inputting the hand motion data and the hand posture data into a motion text generation model, outputting a motion text, inputting the reference description text and the motion text into a text fusion model, and outputting a target description text; the target description text is input into a model generation network, a three-dimensional model is output, and the three-dimensional model comprises at least one entity model and at least one environment scene model corresponding to the entity model; and mapping the three-dimensional model to a medium-free aerial imaging device to obtain an aerial suspension image. The problems of low aerial imaging efficiency, lack of dynamic interaction capability and the like of the existing aerial imaging scheme are solved.

The invention provides a digital twinning-combined dexterous hand teleoperation hand action migration method. The method comprises the following steps: acquiring an image sequence of a hand through a binocular camera; for each group of images in the image sequence, determining a three-dimensional coordinate sequence of the hand feature points; based on a pre-established digital twin skeleton model of the dexterous hand, establishing a mapping relation between the hand feature points and corresponding points in the digital twin skeleton model; converting the three-dimensional coordinate sequence of the hand feature point into a three-dimensional coordinate sequence of a corresponding mapping point in the digital twin skeleton model based on the mapping relation, taking the three-dimensional coordinate sequence as the position of the mapped hand feature point, and determining an optimal buckling angle and an optimal abduction angle; and based on the optimal buckling angle and the optimal abduction angle, a joint angle sequence is obtained, and corresponding joint actions in the dexterous hand are controlled to enable the dexterous hand to perform action migration according to hand actions in the image sequence, so that the smoothness and precision of the actions of the dexterous hand in the action migration process are improved.

The invention discloses a bionic finger, a hand motion control device, a bionic hand control method and a robot, and belongs to the technical field of bionic robotics.The device comprises a plurality of flexible electric actuators which comprise electric actuating materials capable of actively deforming under excitation of an electric field; one end of the flexible electric actuator serves as a fixed end to be connected to the finger adaptation base, and the other end of the flexible electric actuator serves as a free end to be connected to the knuckle structure through a force transmission path. Wherein each flexible electric actuator can be independently controlled, and the plurality of flexible electric actuators cooperatively drive the knuckle structure to generate bionic motion; the passive constraint structure is configured to provide anisotropic mechanical constraint for active deformation of the flexible electric actuator, so that the active deformation is guided and converted into driving displacement or driving force in the first preset direction; and the guide structure is used for guiding the deformation direction of the flexible electric actuator. The device can integrate a strain sensor, an angle sensor and a touch sensor, and realizes closed-loop cooperative control through a controller.

The invention relates to the technical field of dexterous hand motion planning, in particular to a dexterous hand motion planning method and system based on deep learning, and the method comprises the steps: obtaining the state information of a dexterous hand, carrying out the multi-modal feature fusion to generate environment perception information, carrying out the dynamic obstacle detection and cooperative signal analysis, generating an obstacle avoidance path, and optimizing the motion path planning. Real-time data are collected through the sensor module, multi-modal features are fused, and environment sensing information is generated to support motion planning of the dexterous hand; collaborative decision-making among multiple dexterous hands is realized by utilizing an internal communication module, and the adaptive ability and task execution efficiency in a complex dynamic environment are remarkably improved. In addition, dynamic obstacle information is shared by prompting a cooperation signal, conflicts are avoided, and the response speed is increased. The method is suitable for efficient motion planning of the dexterous hand in a complex dynamic scene.

The invention relates to the technical field of intelligent control, and discloses a dexterous hand motion planning and control method based on a visual language motion model, and the method comprises the steps: S1, collecting data, and carrying out the time sequence synchronization and consistency verification of multi-source data; s2, performing dynamic adaptive exponential moving average filtering and calculation processing on the joint sequence, and reconstructing a processing result into adaptive structured features; s3, fusing joint states, inputting the fused joint states into an action head network, splicing online and offline samples into a training batch according to a preset proportion, and determining results to jointly optimize model parameters; s4, performing model reasoning to obtain target action information, performing processing in sequence to generate a smooth low-jitter control sequence, and sending the smooth low-jitter control sequence to the dexterous hand for execution; and S5, the deployment thread issues a control instruction by aligning the annular buffer and the timestamp, triggers a rollback track and allows manual intervention when the control instruction exceeds a threshold value, and writes manual intervention data into an offline buffer. According to the method, efficient, stable and smooth motion control of the dexterous hand can be realized under the condition of extremely few teaching data sets.

The invention discloses a piano teaching interaction method based on a skeleton point image recognition technology. The method comprises the following steps: S1, collecting video data of hand actions when a user plays a piano; s2, carrying out preprocessing on the video data; s3, extracting hand skeleton point data by using the skeleton point extraction model to obtain a skeleton key point data set; s4, training a hand action recognition network model by using the extracted skeleton key point data set and the loss function; and S5, inputting the real-time output of the skeleton point extraction model into the trained hand action recognition network model, and detecting the correctness of the hand posture. According to the method, the motion information of the hands of the piano player can be captured and analyzed in real time with high precision, and targeted and personalized teaching feedback can be realized through deep mining of skeleton point data, so that the interactivity and teaching efficiency of piano teaching are improved.

The invention relates to robot motion control, in particular to a dexterous hand motion algorithm control method, system and device, a storage medium and a program product. The dexterous hand motion algorithm control method comprises the following steps: visual detection: acquiring a hand image through a camera, and identifying three-dimensional coordinates of key points of hand joints by adopting a visual detection algorithm; kinematics calculation: based on the coordinates of the joint key points, obtaining target execution angles of a plurality of motors through kinematics forward and inverse calculation; communication distribution: receiving the target execution angle through a master control unit, and distributing the target execution angle to a plurality of slave control units according to a preset distribution rule; and motor control: each slave control unit generates a driving signal by adopting a PID control algorithm based on the received target execution angle, and controls the corresponding motor to move. The dexterous hand motion algorithm control method provided by the invention is high in response speed, high in control precision, stable in communication and accurate in kinematics solution.

The invention relates to the related technical field of student behavior monitoring, in particular to a classroom student behavior monitoring system, which comprises the following functional modules: an identity label generation module; an image acquisition module; a data storage module; a behavior analysis module; a multi-modal data integration module; and an interaction feedback module. According to the classroom student behavior monitoring system, the identity label generation module, the image acquisition module, the data storage module, the behavior analysis module, the multi-modal data integration module and the interaction feedback module work cooperatively; algorithm design of a head posture analysis unit, a hand action recognition unit, an electronic equipment use detection unit and a sitting posture evaluation unit is further optimized, and a student behavior track diagram and an abnormal condition report are provided as dynamic monitoring results to an external user by means of an interaction feedback module. The defects of the prior art in the aspects of recognition precision, real-time performance, multi-dimensional data analysis and privacy protection are overcome.

The invention discloses a virtual sensory integration training method based on force feedback, and the method comprises the steps: constructing a hot chamber virtual environment containing a mechanical arm model and an operation object through a scene simulation platform, and configuring a joint position according to a real mechanical arm parameter; the dynamics simulation platform builds a corresponding dynamics model and configures inverse kinematics solution and collision detection functions; force feedback equipment is integrated, and a working space mapping model is constructed to convert hand motion into the expected pose of the tail end of the mechanical arm; the double platforms synchronize data through a dynamic link library, and a force sense calculation module calculates collision force and bolt assembling and disassembling torque and transmits the collision force and the bolt assembling and disassembling torque to force feedback equipment. The method does not need to depend on a real mechanical arm, safety risks and high cost are avoided, training repeatability and pertinence are improved, immersion and operation precision are enhanced through force sense and visual sense synchronous feedback, the training period is shortened, and the method is suitable for fine operation training of the mechanical arm in a dangerous environment.

The invention discloses a brain-computer interface manipulator motion detection system for disturbance of consciousness assessment, which belongs to the technical field of disturbance of consciousness assessment and specifically comprises an electroencephalogram acquisition device for acquiring original electroencephalogram signals when a patient executes motor imagery tasks containing different hand action instructions; the dynamic feature extraction module generates a motor imagery activation map by calculating the energy distribution change of a preset frequency band; the driving instruction generation module calculates the space-time matching degree of the map and a preset action instruction and generates a manipulator driving instruction; the manipulator control module is used for controlling and executing a composite action positively correlated with the instruction strength, and the action is formed by combining basic grasping and multi-joint coordination actions in proportion; the motion capture device records track changes, and the ratio of the number of the joints exceeding the threshold value to the motion duration serves as a consciousness activity index; the disturbance of consciousness grading module determines a grading result according to the fluctuation range of the index in the continuous period; the method provides an objective and accurate quantification means for disturbance of consciousness assessment.

The invention discloses a hand motion mapping method based on a data glove, which comprises the following steps of: judging the type of a task to be performed according to data acquired by the data glove, and selecting different mapping methods. If the current gesture is judged to be the gesture representation, applying a gesture recognition model, if the current gesture exists in a gesture library, applying a standard gesture in the library, and if the current gesture does not exist, applying direct joint angle mapping to four fingers except the thumb, and applying Cartesian space mapping to the thumb. And if the task is judged to be the operation task, the data of the data glove is directly processed by a pre-trained neural network and is directly mapped into a joint angle corresponding to the target manipulator. The training thought of the neural network is that the human hand joint angle represented by the data glove and each joint angle of the target manipulator under the same gesture action are obtained at the same time, a neural network model is constructed, the angle of the data glove serves as input, the corresponding angle of the manipulator serves as output, and training is conducted. Finally, the motion mapping process from the hand to the manipulator is achieved.

The invention provides a hand multi-mode information sensing device and a robot humanoid skill learning method, and the device comprises a multi-mode touch sensor module which is used for sensing a hand multi-mode touch signal; the flexible strain sensor module is used for sensing a hand joint bending signal; the multi-modal motion capture module is used for sensing hand motion and posture signals, and the hand motion and posture signals comprise hand motion signals and / or hand posture signals; at least one of the hand multi-mode touch signals, the hand joint bending signals and the hand movement and posture signals forms hand multi-mode information representing gesture actions, hand movement and hand touch. According to the invention, sensing of hand multi-modal information in a hand operation process can be realized based on the hand multi-modal information sensing device, establishment of a sensing strategy and a control strategy is facilitated, cross-object and cross-task robust skill migration is realized, and self-adaptive operation capability and task success rate of a robot in an actual complex environment are remarkably improved.

This invention discloses an adaptive impedance control method and device for a teleoperated robot system. The method is based on a fuzzy logic-based intelligent control mechanism. It dynamically generates a correction update rate for online compensation adaptive terms through fuzzy inference, based on real-time acquired master-slave state information and preset control parameters. This correction update rate, real-time state information, and preset parameters are integrated, and the core dynamic equations of the preset impedance control model are solved to calculate the robot's target position within the current control cycle. This method, based on the robot following the operator's hand movements, superimposes a position compensation amount optimized by the adaptive impedance control model. When problems such as operator hand tremor or network transmission delay cause a sudden increase in contact force, making it impossible to effectively track the desired contact force, the adaptive impedance control device adjusts the contact force, achieving refined teleoperation control.

The application discloses a kind of based on visual perception nimble hand motion planning method, equipment and medium, it is related to robot control technical field, including, based on the multi-modal visual data of real-time acquisition execution object instance segmentation and object six degree of freedom pose estimation, generate the environment state representation including object category, segmentation mask and pose information;When arbitration decision instruction comprehensive risk difference index does not satisfy preset stable execution condition, start global re-planning and generate new planning instruction, and continuously update environment state representation in executable main trajectory execution process.The application is estimated to object six degree of freedom pose by covariance analysis to result and applies counterfactual disturbance, forms the multi-state environment representation around the perception uncertainty in trajectory planning phase, realizes multi-trajectory planning expression under the same task instruction constraint.

The invention discloses a virtual seven-stringed plucked instrument practice system, belongs to the technical field of virtual simulation training, and solves the technical problems of sensing error interference and lack of pertinence and individuation of guidance in existing virtual seven-stringed plucked instrument practice. The system comprises a data acquisition module, a data processing module and a feedback output module. Hand motion sensing data are collected through a data glove, consistency deviation generated by equipment factors and posture deviation generated by user operation are extracted, user types are divided by combining the difference degree of a playing effect and a standard effect and posture deviation characteristics, consistency deviation compensation or compensation and posture correction is executed in a classified mode, and the user performance is improved. Personalized standard hand shape adaptation, dynamic adjustment of a judgment threshold value, deviation pre-judgment and reverse optimization of acquisition sensitivity can also be realized, and a feedback module synchronously outputs virtual images and sound feedback. According to the invention, the accuracy of the sensing data is improved, personalized and real-time Guqin practice guidance is realized, and the Guqin practice experience and practice efficiency are effectively optimized.

The invention relates to the technical field of simulation, in particular to a finger motion trail simulation method and device, equipment, a medium and a program product, and the method comprises the steps: obtaining a hand skeleton image and a hand motion video of a detected target; constructing a three-dimensional hand simulation model according to the hand skeleton image; extracting a plurality of video frames from the hand action video, extracting a two-dimensional coordinate of at least one position point from each of the plurality of video frames, converting the hand action video into a three-dimensional coordinate based on a pre-established coordinate mapping relationship, and forming time sequence data according to the position point of the three-dimensional coordinate; and driving the three-dimensional hand simulation model to execute a simulation action by using the time sequence data, and generating a finger motion track of the tested target according to an execution result of the simulation action. Therefore, the problems of high acquisition cost, poor universality and the like of the finger motion trail in the prior art are solved.

The invention relates to the technical field of action recognition, and provides a hand action recognition method and system based on a lightweight cross-modal space-time network, and the method comprises the following steps: carrying out the preprocessing of a hand-washing video stream, and obtaining an RGB image sequence and a hand key point coordinate sequence; performing separation convolution operation of multi-layer space convolution and time convolution on the RGB image sequence, extracting motion-related features in combination with a channel attention mechanism, and generating a visual feature mark sequence; performing semantic extraction and time sequence compression on the key point coordinate sequence to obtain a structural feature mark sequence aligned with the X; and dynamically calculating a gating coefficient based on context information, fusing X and H to obtain fusion features, and outputting classification probability distribution of the hand washing steps. According to the method, a lightweight visual feature extraction structure is adopted, and a dynamic gating mechanism is introduced to realize adaptive collaborative fusion of visual and structural modes, so that high-precision and low-delay hand action recognition is realized on edge equipment.

The invention provides a driver gesture dynamic detection method and system based on kinematic characteristics. The method comprises the following steps: intercepting a preset number of frame image sequences comprising a whole set of gestures from a video stream in a driver operation process; performing hand detection on each frame of image, and determining coordinates of a hand central point; based on the hand center point coordinates of the two adjacent frames, gesture motion parameters are calculated, and a turning point of hand motion is determined; judging whether an effective hand action exists or not according to the gesture motion parameters and a middle frame of the image sequence, continuing to judge the hand motion direction according to the turning point and the middle frame after judging that the effective hand action exists, and recording a motion direction sequence by adopting a sliding window mode; comparing the motion direction sequence with the direction sequence of the gesture rule, and performing gesture compliance judgment; and when the judgment result is violation, early warning is carried out. The gesture of the driver can be dynamically detected.

The application belongs to the technical field of humanoid robots, and discloses a modular mechanical thumb finger with the motion characteristics of a human thumb, which comprises a base module, a metacarpal module, a proximal phalanx and a distal phalanx. The base module provides a connecting interface with a palm or an arm robot. The metacarpal module has three active degrees of freedom and is driven by a screw block mechanism to accurately realize opposition, abduction and dumping motions. The proximal phalanx realizes bending motion. The distal phalanx is coupled with the proximal phalanx through a four-bar linkage mechanism and is decoupled through a rocker block mechanism, and has passive adaptive capability. The application highly imitates the motion function of a human thumb, has compact structure, efficient driving, and can stably complete fine opposition and gripping operations. Meanwhile, the modular design facilitates integration, and improves the operability, adaptability and intelligent level of the humanoid dexterous hand.

The utility model discloses a detection and evaluation device for a gesture recognition control function of an intelligent range hood. The detection and evaluation device comprises an upper computer, a bionic hand motion control module and an instrument data acquisition module, the bionic hand motion control module comprises a four-degree-of-freedom mechanical intelligent arm and a bionic hand mounted on the four-degree-of-freedom mechanical intelligent arm, and the upper computer is connected with a control end of the four-degree-of-freedom mechanical intelligent arm; the data acquisition module comprises a universal shaping pipe and a pressure gauge installed on the universal shaping pipe, the pressure gauge is supported by the universal shaping pipe and arranged near an air inlet of the range hood, and the data output end of the pressure gauge is connected with the upper computer. The utility model provides a set of solution for the realization device for automatically detecting and evaluating the gesture recognition control function of the intelligent range hood, and the device is high in automation degree, flexible and convenient to control, capable of meeting the diversity of gesture control modes and working logic of products, and suitable for the control requirements of different range hoods.

This invention relates to the field of dexterous hand technology, and provides a dexterous hand thumb structure, a dexterous hand, and a robot. The dexterous hand thumb structure includes a palm drive unit, a first deceleration unit, and a thumb module. By using the palm drive unit and the first deceleration unit as drive components for the metacarpophalangeal joint, the entire thumb module can achieve palmar lateral swinging motion. At the same time, by setting a bending drive unit in the thumb metacarpal joint, the rotation of the CMC joint is controlled by the bending drive unit, thereby enabling the thumb metacarpal joint to swing while simultaneously performing rotational flexion and extension movements. Furthermore, the rotation axis of the thumb metacarpal joint is perpendicular to the rotation axis of the entire thumb module's swinging motion, and the entire movement process is independent and compact. This allows for independent control of thumb bending and palmar movement within a limited space, achieving more precise dexterous hand motion control.

The application belongs to the field of robot control, and particularly relates to a dexterous hand redirection method, equipment and a storage medium. The dexterous hand redirection method comprises the following steps: parameterizing joint angle trajectory of a target dexterous hand into a first parameter tensor to be optimized; constructing a differentiable loss function for constraining consistency of a source hand motion trajectory and a target dexterous hand motion trajectory; iteratively updating the first parameter tensor based on the motion trajectory parameter of the source hand using an optimizer to generate an optimized first parameter tensor; and redirecting the motion trajectory of the target dexterous hand based on the base pose trajectory of the source hand and the optimized first parameter tensor. The method of the application changes from traditional frame-by-frame optimization to global optimization of the entire motion trajectory, the generated motion trajectory is smoother and more coherent, and the system has high integration and good flexibility.

To allow a user to feel a sense when holding or moving an object with strength matched with a situation according to the restriction of an actual operation.SOLUTION: The system includes a display device that presents a video of a virtual space including an object to a user, a measurement device that measures a motion of a hand of the user, a vibration presentation device that presents a vibration to the hand of the user, an audio device that presents a sound to the user, and a control device. The control device controls a plurality of parameters among a visual parameter of a video of the object, a tactile parameter of vibration of the vibration presentation device, and an auditory parameter of sound of the audio device in movement of the object in the virtual space on the basis of an attribute and / or a state of the object.SELECTED DRAWING: Figure 1

The application relates to a dexterous hand multi-modal motion trajectory prediction model generation method and related devices, which are used for simulating a dexterous hand grabbing process, and the method comprises the following steps: generating a motion trajectory of the dexterous hand according to initial position information of the dexterous hand and end position information of a target object; acquiring a multi-modal data set of the dexterous hand in the motion trajectory based on a plurality of motion trajectory points, wherein the multi-modal data set comprises environmental image data, first multi-modal data and second multi-modal data; performing multi-modal feature fusion on the environmental image data, the first multi-modal data and the second multi-modal data to obtain an input sequence; inputting the input sequence into a self-attention model to be trained based on a preset sparse self-attention mechanism to train the self-attention model and obtain a motion trajectory prediction model. According to the scheme, the real-time performance and the accuracy of dexterous hand motion trajectory prediction can be improved through the fusion of multi-modal data, and the grabbing success rate of the dexterous hand is improved.

The invention discloses an intelligent ring early warning system for multi-dimensional hand action monitoring, and relates to the technical field of intelligent rings, and the system comprises a data acquisition module, a gesture extraction module, a behavior judgment module, an early warning module and a personalized module. According to the invention, scene-based and personalized judgment is adopted, and addiction critical points in different situations are accurately adapted, the behavior judgment module calculates behavior diversity indexes based on the distribution of three types of hand operation units and dynamically judges the addiction risk in combination with scene information and a personalized attention baseline, and the personalized module dynamically judges the addiction risk through clustering analysis of historical behaviors of a user. Different use scenes are divided, and the exclusive judgment base line of each scene is continuously updated, so that the system not only can adapt to risk identification of easily addicted scenes, but also can dynamically adjust the judgment standard along with user behavior habit changes, it is ensured that critical points about to be addicted can be accurately perceived in various scenes, misjudgment or missed judgment caused by general standards is avoided, and the user experience is improved. And the early warning accuracy is improved.

The invention discloses a hand movement track recognition method, system and device and a medium. The hand motion trail recognition method comprises the following steps: acquiring hand shot image data; the hand shot image data image is input into an improved YOLOv5 gesture detection model to obtain a hand area, the improved YOLOv5 gesture detection model comprises a backbone network, a neck network and a head network, C3 modules in the backbone network and the neck network are replaced with C3Rep modules, and a SimAM attention mechanism is introduced into the backbone network; and according to the hand region, obtaining a hand motion track. By adopting the embodiment of the invention, the hand region can be quickly positioned, and the hand motion track can be accurately identified and fitted, so that the track identification speed and precision can be improved, and reliable data support can be provided for related technologies.

The application discloses a hand movement ability recovery system based on a brain-computer interface and virtual reality, and relates to the technical field of intelligent interaction control. The system first carries out round-level compilation on electroencephalogram records, line-of-sight records, hand micro-motion records and task state records in the same training process, and obtains a round alignment source data array. Then, a four-source starting anchor point string is formed by extracting a movement intention trigger starting point, a target object stable gaze starting point, a residual hand micro-motion starting point and a task entering an operable stage starting point. Subsequently, a continuous round difference relationship is reversely identified and purified and screened, a sequence purification anchor point string is generated, and a starting misplacement acceptance spectrum is constructed accordingly. A starting offset comparison result is obtained in combination with a reference misplacement spectrum. On this basis, a release motion overlapping window diagram is constructed, a virtual hand motion starting interval is determined, a closed check is performed on a post-round acceptance state after starting, and a time sequence read-back control table is generated.

The application provides a dexterous hand action control method, system and storage medium based on a multi-modal task, and belongs to the technical field of robots. The method comprises the following steps: acquiring image information of a target object in an operation task; identifying shape size features of the target object by analyzing the image information; when the shape type of the target object is irregular or the maximum rotation radius of the target object is greater than the maximum gripping radius of the dexterous hand, performing the operation task in a supporting mode; when the shape type of the target object is regular, the maximum rotation radius of the target object is less than or equal to the maximum gripping radius of the dexterous hand, and the position of the minimum rotation radius is at the top of the target object, performing the operation task in a grabbing mode; and when the shape type of the target object is regular, the maximum rotation radius of the target object is less than or equal to the maximum gripping radius of the dexterous hand, and the position of the minimum rotation radius is at the middle of the target object, performing the operation task in a holding mode. The application improves the adaptability of the dexterous hand to multi-type task targets.