268 results about "Hand motion" patented technology

A multi-modal articulation system translates human gestures into control commands for digital and automation devices. The system integrates a camera module, which captures facial and hand movements, with an image processing unit that extracts facial features, such as eyebrow movement, lip curvature, eyelid motion, and eye trajectory. These extracted features are analyzed in real time by an articulation recognition module that compares them against a database to recognize specific gestures. The system tracks and processes air-drawn hand gestures using a gesture trajectory tracking unit, which converts the motions into digital representations and classifies them using deep learning algorithms. The recognized gestures are converted into machine-readable control signals and executed on various connected devices via a relay control unit. The system incorporates adaptive algorithms to improve accuracy, account for environmental conditions, and stabilize hand tremors, providing a gesture-based control interface for applications ranging from smart home systems to multimedia devices.

The invention discloses a child massage teaching auxiliary system based on artificial intelligence, and particularly relates to the technical field of massage auxiliary teaching, a Kalman filtering technology is adopted to effectively filter out large-amplitude interference in hand motion data, time-frequency analysis is carried out in combination with fast Fourier transform or wavelet transform, and frequency domain characteristics of microfibrillation signals are accurately captured; through fine micro-tremor signal analysis, noise interference is removed, the fine change of micro-tremor of the hands of a student is accurately identified, and the evaluation precision of the massage action stability is remarkably improved; and the integrated learning progress management module automatically adjusts teaching contents according to the learning progress and skill mastering conditions of the students, provides real-time feedback and forward excitation, provides excitation when the students have good performance, prompts and optimizes follow-up actions in real time when deviation occurs, helps the students to continuously improve skills, avoids error accumulation, and effectively improves the teaching effect.

The invention discloses a sign language recognition method and device based on multi-modal deep learning, and the method comprises the steps: multi-modal data input: capturing hand motions, gesture tracks and facial expressions at the same time through a camera, a motion capture sensor and other devices, and forming multi-modal data input; sign language action recognition: precise recognition of sign language actions is realized through a combined model of a deep convolutional neural network and a long-short-term memory network; facial expression and gesture track combined recognition: realizing understanding and translation of complex sign language sentences by combining the captured facial expressions and gesture tracks; and context natural language processing: generating a target statement in combination with context semantic understanding, and outputting a translation result. Through hand motion capture, facial expression analysis, gesture trajectory tracking and context natural language processing, complex sign language motions can be recognized more accurately and translated into characters or voices in real time, and low delay and high accuracy are achieved.

The invention provides a self-supervised continuous 3D hand posture tracking method based on a lightweight inertial measurement unit, and the method specifically comprises the following steps: firstly, obtaining a hand motion signal by using a lightweight consumer-level IMU, and extracting features by using a multi-stage neural network containing a bidirectional long-short term memory (Bi-LSTM) module; the global displacement and orientation of the hand are processed through a wrist posture estimation module, and three-dimensional coordinates of hand skeleton points are solved and calculated by means of a finger motion chain; secondly, a self-supervised completion network is realized by combining a random mask and dual loss, and the features are processed to complete complete attitude reconstruction; and finally, combining network output with kinematics prior, generating a three-dimensional grid with reasonable anatomy, solving the problem that joint deformation is inconsistent with torque, and realizing accurate acquisition of a three-dimensional hand posture. For a dynamic hand motion tracking task, the method can adapt to a sparse IMU deployment scene and realize continuous high-precision tracking; the real-time performance demand of daily interaction can be met, and a low-cost solution can be provided for the fields such as medical rehabilitation and virtual interaction which have strict requirements on attitude precision.

The invention discloses a video stream-based hand hygiene qualified behavior subject identification method for a hand hygiene behavior. The method comprises the following steps of: acquiring a gait, posture and hand action feature image frame sequence; hand position detection and tracking are performed through a target detection model and an optical flow method; classifying hand actions in each image frame through a fine-grained feature extraction model; the effectiveness of the action classification result is judged through a sliding window method, and a final action classification sequence is output; user identity recognition is carried out by fusing the gait, the posture and the hand action features through an identity recognition network, and user identity information is obtained; and finally, counting a comprehensive score of the hand hygiene process in combination with the action classification sequence and the related time, associating the score with the user identity information, and screening out a hand hygiene behavior subject with a qualified evaluation result. According to the invention, high-precision and real-time hand hygiene behavior detection and user identity information identification can be realized, and a more comprehensive scheme is provided for hand hygiene evaluation and management.

The invention discloses a generative interaction method and device for medium-free aerial imaging, and relates to the technical field of man-machine interaction for medium-free aerial imaging. The method comprises the steps of obtaining a reference description text, hand motion data and hand posture data corresponding to a target user, wherein the hand posture data is used for indicating a final posture of hand motion; inputting the hand motion data and the hand posture data into a motion text generation model, outputting a motion text, inputting the reference description text and the motion text into a text fusion model, and outputting a target description text; the target description text is input into a model generation network, a three-dimensional model is output, and the three-dimensional model comprises at least one entity model and at least one environment scene model corresponding to the entity model; and mapping the three-dimensional model to a medium-free aerial imaging device to obtain an aerial suspension image. The problems of low aerial imaging efficiency, lack of dynamic interaction capability and the like of the existing aerial imaging scheme are solved.

The invention provides a digital twinning-combined dexterous hand teleoperation hand action migration method. The method comprises the following steps: acquiring an image sequence of a hand through a binocular camera; for each group of images in the image sequence, determining a three-dimensional coordinate sequence of the hand feature points; based on a pre-established digital twin skeleton model of the dexterous hand, establishing a mapping relation between the hand feature points and corresponding points in the digital twin skeleton model; converting the three-dimensional coordinate sequence of the hand feature point into a three-dimensional coordinate sequence of a corresponding mapping point in the digital twin skeleton model based on the mapping relation, taking the three-dimensional coordinate sequence as the position of the mapped hand feature point, and determining an optimal buckling angle and an optimal abduction angle; and based on the optimal buckling angle and the optimal abduction angle, a joint angle sequence is obtained, and corresponding joint actions in the dexterous hand are controlled to enable the dexterous hand to perform action migration according to hand actions in the image sequence, so that the smoothness and precision of the actions of the dexterous hand in the action migration process are improved.

The invention provides a hand motion hot spot rapid positioning method and device, and the method comprises the steps: obtaining magnetic resonance image data and MEP signal data of an experiment subject, and carrying out the preprocessing of the magnetic resonance image data, and obtaining an electric field simulation result; further constructing a group hand motion hotspot map, and dividing experimental subjects into positive samples and negative samples for training to obtain an outlier classification model; dividing the to-be-positioned testees into an outlier group and a non-outlier group through an outlier classification model; when the to-be-positioned testee is an outlier, performing correlation analysis on an electric field simulation result corresponding to the to-be-positioned testee and the MEP signal data to obtain a target hand motion hotspot; and when the group is a non-outlier group, performing individual hand motion hot spot registration through the group hand motion hot spot map to obtain a target hand motion hot spot. By means of the method and device, the defects that in the prior art, time and labor are wasted in the hand movement hot spot positioning process, positioning accuracy cannot be guaranteed, and consequently clinical treatment efficiency is low are overcome.

The invention discloses a bionic finger, a hand motion control device, a bionic hand control method and a robot, and belongs to the technical field of bionic robotics.The device comprises a plurality of flexible electric actuators which comprise electric actuating materials capable of actively deforming under excitation of an electric field; one end of the flexible electric actuator serves as a fixed end to be connected to the finger adaptation base, and the other end of the flexible electric actuator serves as a free end to be connected to the knuckle structure through a force transmission path. Wherein each flexible electric actuator can be independently controlled, and the plurality of flexible electric actuators cooperatively drive the knuckle structure to generate bionic motion; the passive constraint structure is configured to provide anisotropic mechanical constraint for active deformation of the flexible electric actuator, so that the active deformation is guided and converted into driving displacement or driving force in the first preset direction; and the guide structure is used for guiding the deformation direction of the flexible electric actuator. The device can integrate a strain sensor, an angle sensor and a touch sensor, and realizes closed-loop cooperative control through a controller.

The invention relates to the technical field of dexterous hand motion planning, in particular to a dexterous hand motion planning method and system based on deep learning, and the method comprises the steps: obtaining the state information of a dexterous hand, carrying out the multi-modal feature fusion to generate environment perception information, carrying out the dynamic obstacle detection and cooperative signal analysis, generating an obstacle avoidance path, and optimizing the motion path planning. Real-time data are collected through the sensor module, multi-modal features are fused, and environment sensing information is generated to support motion planning of the dexterous hand; collaborative decision-making among multiple dexterous hands is realized by utilizing an internal communication module, and the adaptive ability and task execution efficiency in a complex dynamic environment are remarkably improved. In addition, dynamic obstacle information is shared by prompting a cooperation signal, conflicts are avoided, and the response speed is increased. The method is suitable for efficient motion planning of the dexterous hand in a complex dynamic scene.

The invention discloses a soft reset type electric piano counterweight keyboard, and relates to the technical field of piano equipment. A key assembly is arranged at the top of the piano shell; a hinge frame is fixedly installed at the bottom in the piano shell through bolts, soft and feedback playing hand feeling is achieved through precise mechanical linkage, meanwhile, the position of an inductance piece is changed by sliding and extruding a clamping bead through a balance weight, and high-precision and high-reliability pressing force detection is achieved in combination with optical sensing auxiliary monitoring. The force signal is converted into keyboard'post-touch layer 'control, and a single-key or global tenuto mode is intelligently distributed; in addition, the key contact state is accurately captured through photoelectric induction, space capture is carried out on hand actions in combination with an infrared camera and a sensing radar, and real-time tactile feedback is provided through electromagnetic driving.

The invention provides a gesture interactive data processing method and device based on AR glasses, and relates to the field of data processing. The method comprises the following steps: acquiring eye movement tracking data and hand action data which are sent by AR glasses and aim at a user; determining sight line direction data of the user according to the eye movement tracking data; determining a gesture feature vector of the user according to the sight line direction data and the hand action data; matching a virtual force field for a virtual object corresponding to the user based on the gesture feature vector; and carrying out dynamic fusion display on the virtual force field and the virtual object through AR glasses. By implementing the technical scheme provided by the invention, the immersion of user operation and the interaction intuition are improved.

The invention relates to the technical field of intelligent control, and discloses a dexterous hand motion planning and control method based on a visual language motion model, and the method comprises the steps: S1, collecting data, and carrying out the time sequence synchronization and consistency verification of multi-source data; s2, performing dynamic adaptive exponential moving average filtering and calculation processing on the joint sequence, and reconstructing a processing result into adaptive structured features; s3, fusing joint states, inputting the fused joint states into an action head network, splicing online and offline samples into a training batch according to a preset proportion, and determining results to jointly optimize model parameters; s4, performing model reasoning to obtain target action information, performing processing in sequence to generate a smooth low-jitter control sequence, and sending the smooth low-jitter control sequence to the dexterous hand for execution; and S5, the deployment thread issues a control instruction by aligning the annular buffer and the timestamp, triggers a rollback track and allows manual intervention when the control instruction exceeds a threshold value, and writes manual intervention data into an offline buffer. According to the method, efficient, stable and smooth motion control of the dexterous hand can be realized under the condition of extremely few teaching data sets.

The invention provides a posture recognition method and device, electronic equipment and a storage medium, and the method comprises the steps: obtaining a to-be-recognized video clip, wherein each image frame of the video clip comprises a hand of a target object; performing feature recognition processing on each image frame to obtain a hand feature map of each image frame; the hand feature map comprises motion space features and motion time sequence features; the motion space feature comprises a hand shape information feature of a hand in the image frame; the motion time sequence features comprise hand motion features of the hand in multiple image frames; and performing classification prediction processing according to the hand feature map of each image frame to obtain sign language text information corresponding to the information transmitted by the hand action in the video clip. In this way, according to the embodiment, the hand shape information and the hand motion information of the target object are concerned, and the recognition ability of the hand posture is improved.

The invention discloses a piano teaching interaction method based on a skeleton point image recognition technology. The method comprises the following steps: S1, collecting video data of hand actions when a user plays a piano; s2, carrying out preprocessing on the video data; s3, extracting hand skeleton point data by using the skeleton point extraction model to obtain a skeleton key point data set; s4, training a hand action recognition network model by using the extracted skeleton key point data set and the loss function; and S5, inputting the real-time output of the skeleton point extraction model into the trained hand action recognition network model, and detecting the correctness of the hand posture. According to the method, the motion information of the hands of the piano player can be captured and analyzed in real time with high precision, and targeted and personalized teaching feedback can be realized through deep mining of skeleton point data, so that the interactivity and teaching efficiency of piano teaching are improved.

A three-dimensional positioning and motion-detecting system for human body and object status in mixed reality and a method thereof are disclosed. A virtualization computing device performs gesture recognition for scaling, moving or 3D rotating a virtual posture model, to make posture model aligning points of the virtual posture model correspond to the posture model positioning points one by one, a virtual-and-real display device displays the adjusted virtual posture model in real time, when the virtual posture model is fixed, the virtualization computing device recognizes a hand position on the physical environment image, and detects a motion on the hand position based on the posture model positioning points, to generate operating position information and hand motion information. The virtual-and-real display device displays virtual-and-real integration of operating position information, hand motion information and a physical space.

The invention relates to robot motion control, in particular to a dexterous hand motion algorithm control method, system and device, a storage medium and a program product. The dexterous hand motion algorithm control method comprises the following steps: visual detection: acquiring a hand image through a camera, and identifying three-dimensional coordinates of key points of hand joints by adopting a visual detection algorithm; kinematics calculation: based on the coordinates of the joint key points, obtaining target execution angles of a plurality of motors through kinematics forward and inverse calculation; communication distribution: receiving the target execution angle through a master control unit, and distributing the target execution angle to a plurality of slave control units according to a preset distribution rule; and motor control: each slave control unit generates a driving signal by adopting a PID control algorithm based on the received target execution angle, and controls the corresponding motor to move. The dexterous hand motion algorithm control method provided by the invention is high in response speed, high in control precision, stable in communication and accurate in kinematics solution.

The invention relates to the related technical field of student behavior monitoring, in particular to a classroom student behavior monitoring system, which comprises the following functional modules: an identity label generation module; an image acquisition module; a data storage module; a behavior analysis module; a multi-modal data integration module; and an interaction feedback module. According to the classroom student behavior monitoring system, the identity label generation module, the image acquisition module, the data storage module, the behavior analysis module, the multi-modal data integration module and the interaction feedback module work cooperatively; algorithm design of a head posture analysis unit, a hand action recognition unit, an electronic equipment use detection unit and a sitting posture evaluation unit is further optimized, and a student behavior track diagram and an abnormal condition report are provided as dynamic monitoring results to an external user by means of an interaction feedback module. The defects of the prior art in the aspects of recognition precision, real-time performance, multi-dimensional data analysis and privacy protection are overcome.

The present invention provides a sign language recognition method and system based on a ConvLSTM model. The method includes: obtaining strain signal data under different hand movements, and performing semantic information annotation on the strain signal data to obtain a data set; performing multi-scale spatio-temporal decomposition on the strain signal data according to a preset multi-dimensional hierarchical transformation (MHT), obtaining high-frequency components and low-frequency components according to the decomposition result, and reconstructing the high-frequency components and low-frequency components to obtain reconstructed features; using the reconstructed features as the input of an improved ConvLSTM model, and using the annotated semantic information as the output of the improved ConvLSTM model, and performing iterative training on the improved ConvLSTM model to obtain a sign language recognition model; inputting the target strain signal data of the hand movement to be recognized into the sign language recognition model to obtain the semantic information corresponding to the target strain signal data. The present invention can improve the recognition accuracy of sign language.

The invention discloses a virtual sensory integration training method based on force feedback, and the method comprises the steps: constructing a hot chamber virtual environment containing a mechanical arm model and an operation object through a scene simulation platform, and configuring a joint position according to a real mechanical arm parameter; the dynamics simulation platform builds a corresponding dynamics model and configures inverse kinematics solution and collision detection functions; force feedback equipment is integrated, and a working space mapping model is constructed to convert hand motion into the expected pose of the tail end of the mechanical arm; the double platforms synchronize data through a dynamic link library, and a force sense calculation module calculates collision force and bolt assembling and disassembling torque and transmits the collision force and the bolt assembling and disassembling torque to force feedback equipment. The method does not need to depend on a real mechanical arm, safety risks and high cost are avoided, training repeatability and pertinence are improved, immersion and operation precision are enhanced through force sense and visual sense synchronous feedback, the training period is shortened, and the method is suitable for fine operation training of the mechanical arm in a dangerous environment.

The invention discloses a multi-channel gesture recognition method and device, and belongs to the field of rehabilitation. N-channel electromyographic signals during hand action and real-time pressure signals between finger pulps and grabbed objects are collected; the N-channel electromyographic signals are preprocessed according to the real-time pressure signals; the preprocessed N-channel electromyographic signals are integrated into a signal matrix; time domain, frequency domain, time-frequency domain and space correlation features are sequentially extracted from the signal matrix, and all the features of each time frame are spliced according to the time sequence to form a high-dimensional feature matrix; the high-dimensional feature matrix is input into a dynamic space-time diagram convolutional network model, the dynamic space-time diagram convolutional network model carries out gesture and force recognition, gesture classification probability distribution and force classification probability distribution are formed, and a result is output, and through the steps, complex gestures and gestures with different forces can be accurately classified and recognized.

The invention discloses a force feedback teleoperation control system and method based on a continuum surgical actuator, and belongs to the technical field of medical surgical robots. The system comprises a master end driving part and a slave end following operation part; the master end driving part is used for acquiring hand action data of an operator in real time, converting the acquired hand action data into a corresponding control signal, and sending the control signal to the slave end following operation part through a preset communication protocol; the slave end following operation part is used for generating a corresponding driving signal and transmitting the generated driving signal to the servo motor so as to control the operation actuator to move; and in the movement process of the operation executor, the interaction force of the operation executor and the target object is sensed in real time, and the interaction force of the operation executor and the target object is fed back to the main end driving part in real time so that an operator can sense the interaction force. According to the scheme, the teleoperation precision can be guaranteed, and meanwhile the telepresence of a doctor is enhanced.

The invention discloses a dexterous hand motion decoding method based on a collaborative base manifold, which comprises the following steps: acquiring electroencephalogram signal characteristics and corresponding hand motion data, and performing preprocessing after time sequence alignment to obtain a training data set; a dexterous hand motion decoding model is constructed, and in the model, a periodic network module maps the electroencephalogram signal features to a phase manifold to obtain electroencephalogram features with periodic features; the control network module extracts decoupled space-time motion control parameters; the collaborative base recruitment module carries out decoding according to the space-time motion control parameters to obtain corresponding hand motion; constructing total loss, and training the dexterous hand motion decoding model by using the training data set; and extracting electroencephalogram signal characteristics to be predicted, inputting the electroencephalogram signal characteristics to the trained dexterous hand motion decoding model, and decoding to obtain corresponding hand motion. According to the invention, more accurate and fine hand motion decoding can be realized.

The invention discloses a brain-computer interface manipulator motion detection system for disturbance of consciousness assessment, which belongs to the technical field of disturbance of consciousness assessment and specifically comprises an electroencephalogram acquisition device for acquiring original electroencephalogram signals when a patient executes motor imagery tasks containing different hand action instructions; the dynamic feature extraction module generates a motor imagery activation map by calculating the energy distribution change of a preset frequency band; the driving instruction generation module calculates the space-time matching degree of the map and a preset action instruction and generates a manipulator driving instruction; the manipulator control module is used for controlling and executing a composite action positively correlated with the instruction strength, and the action is formed by combining basic grasping and multi-joint coordination actions in proportion; the motion capture device records track changes, and the ratio of the number of the joints exceeding the threshold value to the motion duration serves as a consciousness activity index; the disturbance of consciousness grading module determines a grading result according to the fluctuation range of the index in the continuous period; the method provides an objective and accurate quantification means for disturbance of consciousness assessment.

The invention relates to the technical field of intelligent wearable equipment, and provides a hand motion capture data glove based on an IMU sensor. The thumb mechanism comprises a first thumb section, a second thumb section, a metacarpal bone shell and a connecting piece, the first thumb section and the second thumb section are connected in a sliding and rotating mode, and the metacarpal bone shell is connected with the main control board through the connecting piece; each finger mechanism comprises a first knuckle, a second knuckle, a third knuckle and a telescopic rotating connecting assembly, the first knuckles are in sliding rotating connection with the second knuckles, the second knuckles are in sliding rotating connection with the third knuckles, and the third knuckles are connected with the main control panel through the telescopic rotating connecting assemblies; the first thumb section, the second thumb section, the metacarpal bone shell, the first knuckle, the second knuckle and the third knuckle are each provided with an IMU sensor, each IMU sensor is electrically connected with the main control board, and the main control board is integrated with the IMU sensors. Therefore, production is simple, and cost is low.

The invention discloses a gesture recognition method, device and equipment based on multiple sensors, a medium and a program product, and the method comprises the steps: decomposing multi-sensor data into a plurality of action frames according to a time sequence based on sensor timestamps, and generating an original sequence data set based on the plurality of action frames; the original sequence data set is preprocessed, a hand action time sequence is generated, and preprocessing comprises the step of compensating the original sequence data set based on the environment data; and distributing data weights of the sensors based on the contribution degrees of the sensors, performing feature fusion on the multi-modal hand action data of the time points in the hand action time sequence based on the data weights, performing time-space coding on the fused features, and inputting time-space coding vectors into a gesture recognition model for gesture recognition. According to the method, effective fusion of multi-modal sensor data is realized, and data compensation is performed by combining environmental data, so that the noise influence of the environment on the sensor is effectively reduced, and the gesture recognition precision is greatly improved.

The invention relates to the technical field of artificial intelligence, and discloses a digital human driving method based on artificial intelligence, comprising the following steps: step 1, acquiring a voice signal, a facial electromyographic signal, a pupil size signal and a hand motion signal of a student through a wearable device, the wearable device comprising a helmet built-in sensor and a hand motion capture device; step 2, converting the voice signal in the step 1 into a fundamental frequency feature vector, converting the facial electromyographic signal into a 28-dimensional muscle activation degree vector, converting the pupil size signal into a diameter change rate scalar, and converting the hand motion signal into a six-degree-of-freedom pose matrix; according to the method, the multi-mode behavior recognition engine is adopted to fuse micro expression analysis, voice stress detection and skeleton motion capture technologies, the technical effect of recognizing the tiny abnormal reaction of the student in real time is achieved, and compared with a visual behavior recognition scheme based on a preset script in the prior art, the defects that deviation correction response is delayed and the safety knowledge memory effect is poor are overcome.

The invention provides a fine motor imagery method and system based on hand motion parameter variation, the fine motor imagery method based on hand motion parameter variation can be applied to a cerebral apoplexy rehabilitation experiment / training system, the system comprises an electroencephalogram acquisition device, a motion capture device, a first computer, a second computer and a bearing platform, the fine motion imagination method based on the hand motion parameter change comprises the following steps: S1, building a high-precision motion capture and electroencephalogram synchronous acquisition experiment platform; S2, carrying out a speed-single parameter fine motion task; and S3, carrying out a speed-direction double-vector parameter fine motion task. Based on the fine motion tasks of different motion parameters, the degree of freedom of the motion imagination task is increased, the available normal form of the motion imagination task is expanded, and the brain coding and decoding rule of the vector space motion intention rich in direction, speed and other information can be further understood by adopting the speed-direction fine motion intention experimental design scheme.

The invention discloses a hand motion mapping method based on a data glove, which comprises the following steps of: judging the type of a task to be performed according to data acquired by the data glove, and selecting different mapping methods. If the current gesture is judged to be the gesture representation, applying a gesture recognition model, if the current gesture exists in a gesture library, applying a standard gesture in the library, and if the current gesture does not exist, applying direct joint angle mapping to four fingers except the thumb, and applying Cartesian space mapping to the thumb. And if the task is judged to be the operation task, the data of the data glove is directly processed by a pre-trained neural network and is directly mapped into a joint angle corresponding to the target manipulator. The training thought of the neural network is that the human hand joint angle represented by the data glove and each joint angle of the target manipulator under the same gesture action are obtained at the same time, a neural network model is constructed, the angle of the data glove serves as input, the corresponding angle of the manipulator serves as output, and training is conducted. Finally, the motion mapping process from the hand to the manipulator is achieved.