332 results about "Hand motion" patented technology

A multi-modal articulation system translates human gestures into control commands for digital and automation devices. The system integrates a camera module, which captures facial and hand movements, with an image processing unit that extracts facial features, such as eyebrow movement, lip curvature, eyelid motion, and eye trajectory. These extracted features are analyzed in real time by an articulation recognition module that compares them against a database to recognize specific gestures. The system tracks and processes air-drawn hand gestures using a gesture trajectory tracking unit, which converts the motions into digital representations and classifies them using deep learning algorithms. The recognized gestures are converted into machine-readable control signals and executed on various connected devices via a relay control unit. The system incorporates adaptive algorithms to improve accuracy, account for environmental conditions, and stabilize hand tremors, providing a gesture-based control interface for applications ranging from smart home systems to multimedia devices.

The invention provides a cloud game virtual reality gesture interaction method and device based on self-adaptive multi-modal fusion and a storage medium. The method comprises the following steps that S1, hand skeleton point data of a user is collected, three-dimensional coordinates of key skeleton points are obtained, and skeleton point feature vectors are generated; s2, collecting hand depth map data, extracting depth gradient features of a hand region, and generating a depth gradient feature vector; s3, collecting time sequence data of hand movement, calculating speed and acceleration characteristics of key skeleton points, and generating a time sequence characteristic vector; s4, performing weighted fusion on the skeleton point features, the depth map features and the time sequence features to generate a fusion feature vector; s5, inferring a user interaction intention and executing corresponding virtual reality operation in combination with the fusion feature vector and the scene information; through weighted fusion of multi-modal features, the method significantly improves the accuracy and anti-interference capability of gesture recognition, and is suitable for cloud games and virtual reality interaction scenes.

The invention discloses a child massage teaching auxiliary system based on artificial intelligence, and particularly relates to the technical field of massage auxiliary teaching, a Kalman filtering technology is adopted to effectively filter out large-amplitude interference in hand motion data, time-frequency analysis is carried out in combination with fast Fourier transform or wavelet transform, and frequency domain characteristics of microfibrillation signals are accurately captured; through fine micro-tremor signal analysis, noise interference is removed, the fine change of micro-tremor of the hands of a student is accurately identified, and the evaluation precision of the massage action stability is remarkably improved; and the integrated learning progress management module automatically adjusts teaching contents according to the learning progress and skill mastering conditions of the students, provides real-time feedback and forward excitation, provides excitation when the students have good performance, prompts and optimizes follow-up actions in real time when deviation occurs, helps the students to continuously improve skills, avoids error accumulation, and effectively improves the teaching effect.

The invention discloses a sign language recognition method and device based on multi-modal deep learning, and the method comprises the steps: multi-modal data input: capturing hand motions, gesture tracks and facial expressions at the same time through a camera, a motion capture sensor and other devices, and forming multi-modal data input; sign language action recognition: precise recognition of sign language actions is realized through a combined model of a deep convolutional neural network and a long-short-term memory network; facial expression and gesture track combined recognition: realizing understanding and translation of complex sign language sentences by combining the captured facial expressions and gesture tracks; and context natural language processing: generating a target statement in combination with context semantic understanding, and outputting a translation result. Through hand motion capture, facial expression analysis, gesture trajectory tracking and context natural language processing, complex sign language motions can be recognized more accurately and translated into characters or voices in real time, and low delay and high accuracy are achieved.

The invention provides a self-supervised continuous 3D hand posture tracking method based on a lightweight inertial measurement unit, and the method specifically comprises the following steps: firstly, obtaining a hand motion signal by using a lightweight consumer-level IMU, and extracting features by using a multi-stage neural network containing a bidirectional long-short term memory (Bi-LSTM) module; the global displacement and orientation of the hand are processed through a wrist posture estimation module, and three-dimensional coordinates of hand skeleton points are solved and calculated by means of a finger motion chain; secondly, a self-supervised completion network is realized by combining a random mask and dual loss, and the features are processed to complete complete attitude reconstruction; and finally, combining network output with kinematics prior, generating a three-dimensional grid with reasonable anatomy, solving the problem that joint deformation is inconsistent with torque, and realizing accurate acquisition of a three-dimensional hand posture. For a dynamic hand motion tracking task, the method can adapt to a sparse IMU deployment scene and realize continuous high-precision tracking; the real-time performance demand of daily interaction can be met, and a low-cost solution can be provided for the fields such as medical rehabilitation and virtual interaction which have strict requirements on attitude precision.

The invention discloses a video stream-based hand hygiene qualified behavior subject identification method for a hand hygiene behavior. The method comprises the following steps of: acquiring a gait, posture and hand action feature image frame sequence; hand position detection and tracking are performed through a target detection model and an optical flow method; classifying hand actions in each image frame through a fine-grained feature extraction model; the effectiveness of the action classification result is judged through a sliding window method, and a final action classification sequence is output; user identity recognition is carried out by fusing the gait, the posture and the hand action features through an identity recognition network, and user identity information is obtained; and finally, counting a comprehensive score of the hand hygiene process in combination with the action classification sequence and the related time, associating the score with the user identity information, and screening out a hand hygiene behavior subject with a qualified evaluation result. According to the invention, high-precision and real-time hand hygiene behavior detection and user identity information identification can be realized, and a more comprehensive scheme is provided for hand hygiene evaluation and management.

Disclosed in embodiments of the present description is a sign language recognition method, comprising: acquiring a video stream to be recognized, said video stream comprising continuous sign language action image sequences; acquiring a hand motion posture sequence corresponding to each sign language action image sequence in said video stream; inputting said video stream into a pre-trained first recognition model to obtain a first recognition result, the first recognition result comprising first probability distribution of each target word in a preset word list; inputting the hand motion posture sequence into a pre-trained second recognition model to obtain a second recognition result, the second recognition result comprising second probability distribution of each target word in the word list; and determining a target text on the basis of the first recognition result and the second recognition result. Correspondingly, disclosed in the present invention are a sign language recognition apparatus, and a corresponding interaction system and electronic device.

The invention discloses a generative interaction method and device for medium-free aerial imaging, and relates to the technical field of man-machine interaction for medium-free aerial imaging. The method comprises the steps of obtaining a reference description text, hand motion data and hand posture data corresponding to a target user, wherein the hand posture data is used for indicating a final posture of hand motion; inputting the hand motion data and the hand posture data into a motion text generation model, outputting a motion text, inputting the reference description text and the motion text into a text fusion model, and outputting a target description text; the target description text is input into a model generation network, a three-dimensional model is output, and the three-dimensional model comprises at least one entity model and at least one environment scene model corresponding to the entity model; and mapping the three-dimensional model to a medium-free aerial imaging device to obtain an aerial suspension image. The problems of low aerial imaging efficiency, lack of dynamic interaction capability and the like of the existing aerial imaging scheme are solved.

The invention provides an AI-based gesture correction method and device, and the method comprises the following steps: S1, data collection: capturing the hand motion and posture of a user through a high-precision sensor or camera; s2, gesture recognition: analyzing the collected hand actions and gestures through an AI algorithm, and recognizing the types of gestures; s3, action analysis: comparing the recognized gesture with a preset standard gesture by using a vision algorithm and a posture recognition algorithm, and evaluating the fluency and accuracy of the gesture; s4, real-time feedback: giving real-time feedback to the user through a visual interface or audio prompt according to the result of the action analysis; and S5, correcting suggestions: automatically generating the correcting suggestions when the gesture is found to be inaccurate through action analysis. The AI-based gesture correction method and device provided by the invention aim at improving the efficiency of embroidery teaching and creation through an intelligent technology.

The invention provides a digital twinning-combined dexterous hand teleoperation hand action migration method. The method comprises the following steps: acquiring an image sequence of a hand through a binocular camera; for each group of images in the image sequence, determining a three-dimensional coordinate sequence of the hand feature points; based on a pre-established digital twin skeleton model of the dexterous hand, establishing a mapping relation between the hand feature points and corresponding points in the digital twin skeleton model; converting the three-dimensional coordinate sequence of the hand feature point into a three-dimensional coordinate sequence of a corresponding mapping point in the digital twin skeleton model based on the mapping relation, taking the three-dimensional coordinate sequence as the position of the mapped hand feature point, and determining an optimal buckling angle and an optimal abduction angle; and based on the optimal buckling angle and the optimal abduction angle, a joint angle sequence is obtained, and corresponding joint actions in the dexterous hand are controlled to enable the dexterous hand to perform action migration according to hand actions in the image sequence, so that the smoothness and precision of the actions of the dexterous hand in the action migration process are improved.

The invention provides a hand motion hot spot rapid positioning method and device, and the method comprises the steps: obtaining magnetic resonance image data and MEP signal data of an experiment subject, and carrying out the preprocessing of the magnetic resonance image data, and obtaining an electric field simulation result; further constructing a group hand motion hotspot map, and dividing experimental subjects into positive samples and negative samples for training to obtain an outlier classification model; dividing the to-be-positioned testees into an outlier group and a non-outlier group through an outlier classification model; when the to-be-positioned testee is an outlier, performing correlation analysis on an electric field simulation result corresponding to the to-be-positioned testee and the MEP signal data to obtain a target hand motion hotspot; and when the group is a non-outlier group, performing individual hand motion hot spot registration through the group hand motion hot spot map to obtain a target hand motion hot spot. By means of the method and device, the defects that in the prior art, time and labor are wasted in the hand movement hot spot positioning process, positioning accuracy cannot be guaranteed, and consequently clinical treatment efficiency is low are overcome.

The invention discloses a bionic finger, a hand motion control device, a bionic hand control method and a robot, and belongs to the technical field of bionic robotics.The device comprises a plurality of flexible electric actuators which comprise electric actuating materials capable of actively deforming under excitation of an electric field; one end of the flexible electric actuator serves as a fixed end to be connected to the finger adaptation base, and the other end of the flexible electric actuator serves as a free end to be connected to the knuckle structure through a force transmission path. Wherein each flexible electric actuator can be independently controlled, and the plurality of flexible electric actuators cooperatively drive the knuckle structure to generate bionic motion; the passive constraint structure is configured to provide anisotropic mechanical constraint for active deformation of the flexible electric actuator, so that the active deformation is guided and converted into driving displacement or driving force in the first preset direction; and the guide structure is used for guiding the deformation direction of the flexible electric actuator. The device can integrate a strain sensor, an angle sensor and a touch sensor, and realizes closed-loop cooperative control through a controller.

The invention relates to the technical field of dexterous hand motion planning, in particular to a dexterous hand motion planning method and system based on deep learning, and the method comprises the steps: obtaining the state information of a dexterous hand, carrying out the multi-modal feature fusion to generate environment perception information, carrying out the dynamic obstacle detection and cooperative signal analysis, generating an obstacle avoidance path, and optimizing the motion path planning. Real-time data are collected through the sensor module, multi-modal features are fused, and environment sensing information is generated to support motion planning of the dexterous hand; collaborative decision-making among multiple dexterous hands is realized by utilizing an internal communication module, and the adaptive ability and task execution efficiency in a complex dynamic environment are remarkably improved. In addition, dynamic obstacle information is shared by prompting a cooperation signal, conflicts are avoided, and the response speed is increased. The method is suitable for efficient motion planning of the dexterous hand in a complex dynamic scene.

The humanoid dexterous finger comprises a single-finger base, a fingertip, a first knuckle and a second knuckle, the fingertip, the first knuckle and the second knuckle are sequentially hinged from top to bottom, the bottom end of the second knuckle is rotationally connected with the single-finger base through a swing part, and the rotating plane, relative to the swing part, of the second knuckle is not parallel to the rotating plane, relative to the single-finger base, of the swing part; the fingertips, the first knuckles and the second knuckles are respectively connected with at least one group of connecting rod driving mechanisms for driving the fingertips, the first knuckles and the second knuckles to independently act; each connecting rod driving mechanism comprises a main connecting rod and a driving assembly driving the main connecting rod to move, each driving assembly comprises a power piece, and the power pieces drive the connecting seats to slide back and forth along the sliding rails through pull ropes. The humanoid dexterous hand comprises a palm and the humanoid dexterous finger which is arranged on the palm and adopts the scheme. The humanoid robot adopts the humanoid dexterous hand. The dexterous finger has four active degrees of freedom and higher response speed, and hand actions closer to a real person are realized.

According to the basketball shooting training system and device based on intelligent image analysis, the hand detection module, the gesture key point detection module, the hand shape analysis module, the shooting track simulation module, the intelligent feedback generation module, the data management module, the user interaction interface module and the like are integrated, and the system analyzes hand actions of a trainer through videos, detects joint points and compares the hand actions with standard gestures; according to the device, the hand shape difference is evaluated, a physical simulation algorithm is utilized, a basketball flight path is simulated according to a hand shape analysis result, hit rate prediction is provided, an intelligent feedback module generates personalized training suggestions according to a simulation result and historical data, and the whole device helps trainees to optimize shooting skills and improve the training effect through real-time data and visual feedback.

The invention discloses a soft reset type electric piano counterweight keyboard, and relates to the technical field of piano equipment. A key assembly is arranged at the top of the piano shell; a hinge frame is fixedly installed at the bottom in the piano shell through bolts, soft and feedback playing hand feeling is achieved through precise mechanical linkage, meanwhile, the position of an inductance piece is changed by sliding and extruding a clamping bead through a balance weight, and high-precision and high-reliability pressing force detection is achieved in combination with optical sensing auxiliary monitoring. The force signal is converted into keyboard'post-touch layer 'control, and a single-key or global tenuto mode is intelligently distributed; in addition, the key contact state is accurately captured through photoelectric induction, space capture is carried out on hand actions in combination with an infrared camera and a sensing radar, and real-time tactile feedback is provided through electromagnetic driving.

The invention discloses a method for generating a hand posture action sequence by music based on a diffusion model. The method comprises the following steps: firstly, performing feature extraction on an audio through a pre-trained audio encoder to obtain a feature vector corresponding to the audio; learning position codes based on audio features, and obtaining a corresponding hand position through a hand position predictor; a time step code and a hand posture decoder are introduced, a diffusion model is utilized, and the relation between audio features and hand postures is learned in the noise adding and denoising process. According to the method, through the steps of audio feature coding, hand position decoding, posture embedding and the like, generation of a sequence from music to hand posture actions is achieved, hand actions which are synchronous with the music and have rich details can be generated, and the method is efficient and has good robustness.

The invention discloses a post-stroke hand movement function rehabilitation state evaluation system and method and a training system, and relates to the field of medical rehabilitation, the evaluation system comprises a signal acquisition device, a communication device and an upper computer; the training system comprises an evaluation system and a virtual reality device. The signal acquisition device acquires multi-modal motion signals of a stroke patient completing hand rehabilitation training, a multi-modal hand motion state evaluation model is deployed in the upper computer and used for preprocessing the multi-modal motion signals, inputting the preprocessed multi-modal motion signals into the multi-modal hand motion state evaluation model and outputting a hand motion state score prediction value, and the score prediction value is used for evaluating the hand motion state of the stroke patient. And performing hand motion function rehabilitation state evaluation according to the hand motion state score prediction value. According to the invention, the accuracy of evaluating the rehabilitation state of the hand movement function can be improved.

The invention provides a gesture interactive data processing method and device based on AR glasses, and relates to the field of data processing. The method comprises the following steps: acquiring eye movement tracking data and hand action data which are sent by AR glasses and aim at a user; determining sight line direction data of the user according to the eye movement tracking data; determining a gesture feature vector of the user according to the sight line direction data and the hand action data; matching a virtual force field for a virtual object corresponding to the user based on the gesture feature vector; and carrying out dynamic fusion display on the virtual force field and the virtual object through AR glasses. By implementing the technical scheme provided by the invention, the immersion of user operation and the interaction intuition are improved.

The invention relates to the technical field of intelligent control, and discloses a dexterous hand motion planning and control method based on a visual language motion model, and the method comprises the steps: S1, collecting data, and carrying out the time sequence synchronization and consistency verification of multi-source data; s2, performing dynamic adaptive exponential moving average filtering and calculation processing on the joint sequence, and reconstructing a processing result into adaptive structured features; s3, fusing joint states, inputting the fused joint states into an action head network, splicing online and offline samples into a training batch according to a preset proportion, and determining results to jointly optimize model parameters; s4, performing model reasoning to obtain target action information, performing processing in sequence to generate a smooth low-jitter control sequence, and sending the smooth low-jitter control sequence to the dexterous hand for execution; and S5, the deployment thread issues a control instruction by aligning the annular buffer and the timestamp, triggers a rollback track and allows manual intervention when the control instruction exceeds a threshold value, and writes manual intervention data into an offline buffer. According to the method, efficient, stable and smooth motion control of the dexterous hand can be realized under the condition of extremely few teaching data sets.

The invention provides a posture recognition method and device, electronic equipment and a storage medium, and the method comprises the steps: obtaining a to-be-recognized video clip, wherein each image frame of the video clip comprises a hand of a target object; performing feature recognition processing on each image frame to obtain a hand feature map of each image frame; the hand feature map comprises motion space features and motion time sequence features; the motion space feature comprises a hand shape information feature of a hand in the image frame; the motion time sequence features comprise hand motion features of the hand in multiple image frames; and performing classification prediction processing according to the hand feature map of each image frame to obtain sign language text information corresponding to the information transmitted by the hand action in the video clip. In this way, according to the embodiment, the hand shape information and the hand motion information of the target object are concerned, and the recognition ability of the hand posture is improved.

The invention discloses a hand motion generation method and device for musical instrument playing and a medium. The method comprises the steps of obtaining a target audio clip; inputting the target audio clip into the basic model, and extracting the target audio clip by using an audio detail extractor to obtain audio features; performing hand position prediction on the audio features by using a hand position predictor to obtain a hand prediction position sequence; and inputting the target audio clip and the hand prediction position sequence into a gesture generator for multi-step inverse diffusion processing to obtain de-noised gesture features, and obtaining a hand motion diagram output by the basic model based on the hand prediction position sequence and the de-noised gesture features. According to the invention, the accuracy of generating the hand motion diagram can be improved.

A training bat for training athletes in sports such as baseball and softball is provided. The bat provides a significantly reduced moment of inertia for a given weight of bat. The combination of high weight and low moment of inertia requires significantly exertion to move the bat forwards through a swing without requiring excessive exertion to rotate the bat through the swing. The training bat allows for swing strength and speed while simultaneously developing good hand motion to rotate the bat.

The embodiment of the invention provides a shooting system, method and equipment of intelligent glasses, a medium and a product. The method comprises the following steps: monitoring the hand action of a user in real time through a bracelet, determining whether the user starts to raise the hand or not in real time according to the hand action monitored in real time, predicting that the user has a shooting demand at the moment when determining that the user starts to raise the hand, generating a non-inductive shooting instruction, and sending the non-inductive shooting instruction to connected intelligent glasses, and executing non-inductive shooting based on the non-inductive shooting instruction through the intelligent glasses to obtain a non-delay picture, storing the non-delay picture in a picture library, and waiting for receiving a formal shooting instruction initiated by user action interaction within a preset time period so as to update the picture library according to a receiving state. The method is used for improving the shooting response speed of the intelligent glasses, the probability of shooting the target picture is improved, and the user experience is enhanced.

The invention discloses a piano teaching interaction method based on a skeleton point image recognition technology. The method comprises the following steps: S1, collecting video data of hand actions when a user plays a piano; s2, carrying out preprocessing on the video data; s3, extracting hand skeleton point data by using the skeleton point extraction model to obtain a skeleton key point data set; s4, training a hand action recognition network model by using the extracted skeleton key point data set and the loss function; and S5, inputting the real-time output of the skeleton point extraction model into the trained hand action recognition network model, and detecting the correctness of the hand posture. According to the method, the motion information of the hands of the piano player can be captured and analyzed in real time with high precision, and targeted and personalized teaching feedback can be realized through deep mining of skeleton point data, so that the interactivity and teaching efficiency of piano teaching are improved.

A three-dimensional positioning and motion-detecting system for human body and object status in mixed reality and a method thereof are disclosed. A virtualization computing device performs gesture recognition for scaling, moving or 3D rotating a virtual posture model, to make posture model aligning points of the virtual posture model correspond to the posture model positioning points one by one, a virtual-and-real display device displays the adjusted virtual posture model in real time, when the virtual posture model is fixed, the virtualization computing device recognizes a hand position on the physical environment image, and detects a motion on the hand position based on the posture model positioning points, to generate operating position information and hand motion information. The virtual-and-real display device displays virtual-and-real integration of operating position information, hand motion information and a physical space.

The invention discloses a wearable device-based non-inductive vehicle door unlocking method and system and a vehicle, and the method comprises the steps: obtaining the distance between a wearable device and the vehicle, and building the communication connection between the wearable device and the vehicle when the distance between the wearable device and the vehicle is smaller than a first preset distance; when the distance between the wearable device and the vehicle is smaller than a second preset distance, acquiring a real-time hand movement track of the user; comparing the real-time hand movement track with a preset reference hand movement track data set; when the real-time hand movement track is matched with the reference hand movement track data set, the vehicle door is unlocked; when the real-time hand movement track is not matched with the reference hand movement track data set, reminding a user of unlocking failure; according to the real-time hand motion trail, the reference hand motion trail data set is corrected, a user can complete unlocking of the vehicle door through the subconscious motion of the user under the condition that other redundant motion is not carried out, and the convenience and safety of unlocking the vehicle door are enhanced.

The invention provides a dexterous manipulator fused with an ontology visual perception function, and belongs to the technical field of electromechanical control. The depth camera is arranged in the palm, the size of the device is remarkably reduced, the size of the device is closer to the size of a human hand, and therefore flexibility and naturalness are improved, the depth camera and a control system are highly integrated, the structure is simplified, synchronism of visual data and hand actions is improved, operation precision is improved, the dexterous manipulator has 16 degrees of freedom, and the flexibility and the reliability of the device are improved. The system is completely driven by a steering engine, the operation flexibility and the fine action execution capability are greatly improved, the requirements of complex tasks are met, the production and maintenance cost is effectively reduced through optimization design and reduction of external accessories, multiple objects are positioned at the same time through a developed instance-level multi-target positioning method and by using a neural network, pre-training is not needed, and the operation efficiency is improved. The space perception capability of the dexterous hand is further improved, and the autonomous operation capability of the dexterous hand in a complex environment is enhanced.

The invention relates to robot motion control, in particular to a dexterous hand motion algorithm control method, system and device, a storage medium and a program product. The dexterous hand motion algorithm control method comprises the following steps: visual detection: acquiring a hand image through a camera, and identifying three-dimensional coordinates of key points of hand joints by adopting a visual detection algorithm; kinematics calculation: based on the coordinates of the joint key points, obtaining target execution angles of a plurality of motors through kinematics forward and inverse calculation; communication distribution: receiving the target execution angle through a master control unit, and distributing the target execution angle to a plurality of slave control units according to a preset distribution rule; and motor control: each slave control unit generates a driving signal by adopting a PID control algorithm based on the received target execution angle, and controls the corresponding motor to move. The dexterous hand motion algorithm control method provided by the invention is high in response speed, high in control precision, stable in communication and accurate in kinematics solution.

The invention relates to the related technical field of student behavior monitoring, in particular to a classroom student behavior monitoring system, which comprises the following functional modules: an identity label generation module; an image acquisition module; a data storage module; a behavior analysis module; a multi-modal data integration module; and an interaction feedback module. According to the classroom student behavior monitoring system, the identity label generation module, the image acquisition module, the data storage module, the behavior analysis module, the multi-modal data integration module and the interaction feedback module work cooperatively; algorithm design of a head posture analysis unit, a hand action recognition unit, an electronic equipment use detection unit and a sitting posture evaluation unit is further optimized, and a student behavior track diagram and an abnormal condition report are provided as dynamic monitoring results to an external user by means of an interaction feedback module. The defects of the prior art in the aspects of recognition precision, real-time performance, multi-dimensional data analysis and privacy protection are overcome.