37 results about "Gesture classification" patented technology

Systems and methods for content processing can include obtaining a gesture input and display data, determining content selected by the gesture input, classifying the gesture, and performing a particular data processing action based on the content selection and the gesture classification. The particular data processing action can vary based on gesture classification. The content selection determination can include determining a gesture mask and then determining the features of the displayed content item that are within the gesture mask.

The invention relates to the technical field of computer vision and human-computer interaction, and discloses a self-adaptive hand image recognition method and system based on multi-source feature fusion. The method comprises the following steps: acquiring a real-time video stream image from a camera, carrying out various filtering processing on the real-time video stream image, and respectively generating an illumination enhanced image, a background subtraction mask, a skin color area mask and a dynamic motion mask; dynamically adjusting the fusion weight of each filtering processing result according to the scene features of the real-time video stream image, and generating comprehensive filtering output through weighted fusion; performing multi-stage hand segmentation processing on the comprehensive filtering output to obtain an initial hand region mask, performing time sequence smoothing processing, and extracting a maximum connected domain as a final hand region mask; and inputting the final hand region mask into a convolutional neural network for gesture recognition, outputting a prediction probability of each type of gestures, and displaying a gesture classification result in real time. According to the method, the adaptability of hand detection to strong interference backgrounds and variable environments is enhanced.

Systems and methods for content processing can include obtaining a gesture input and display data, determining content selected by the gesture input, classifying the gesture, and performing a particular data processing action based on the content selection and the gesture classification. The particular data processing action can vary based on gesture classification. The content selection determination can include determining a gesture mask and then determining the features of the displayed content item that are within the gesture mask.

Systems and methods for content processing can include obtaining a gesture input and display data, determining content selected by the gesture input, classifying the gesture, and performing a particular data processing action based on the content selection and the gesture classification. The particular data processing action can vary based on gesture classification. The content selection determination can include determining a gesture mask and then determining the features of the displayed content item that are within the gesture mask.

Methods and apparatuses (devices, methods, etc.) for determining gestures, e.g., hand gestures, from a wrist and / or forearm worn array of magnetometers. These methods and apparatuses may use a trained machine-learning agent to identify gestures from magnetic signals that have been preprocessed to optimize gesture detection. The trained machine learning agent may apply a hierarchical classification that first identifies a first category of gestures from a second category of gestures and then classifies between the individual gestures within each category. The methods and apparatuses described herein may be of particular use with acoustically driven ferromagnetic resonance (ADFMR) sensors.

The embodiment of the invention discloses a gesture recognition method and electronic equipment. The method comprises the following steps: performing image acquisition on a user gesture through an image acquisition device to obtain a continuous video sequence; performing gesture region extraction on each image frame in the continuous video sequence to obtain a gesture image, and performing feature extraction on the gesture image to obtain spatial stream features; performing optical flow calculation on two continuous frames of images to obtain optical flow images, and performing feature extraction on the optical flow images to obtain time flow features; and fusing the spatial flow features and the time flow features to obtain fused features, and performing gesture classification on the fused features based on a classifier to determine a gesture recognition result. According to the scheme, the space features and the time features can be subjected to cross-modal correlation fusion, so that the two features are complementary in gesture recognition, the motion features can know that the apparent features pay attention to a dynamic area, the border features can correct misjudgment of the operation features, and the accuracy of gesture recognition is improved.

The invention relates to the technical field of electromyographic signal processing, in particular to an electromyographic acquisition signal processing method and system for gesture classification. The method comprises the following steps: forming a signal label and an electromyographic signal sample cluster based on a gesture type, electromyographic signal acquisition duration and gesture recognition accuracy; calibrating a cross-cluster error value of the electromyographic signal features; mapping the cross-cluster error value to a two-dimensional coordinate system and adding a point feature value; and generating a to-be-processed sample set through the dual-scale data screening window, judging a gesture type membership degree, and executing visual display. According to the method, the classification precision problem caused by user action habit differences is solved through intelligent data analysis, the accuracy and adaptability of gesture classification are improved, and the method is suitable for scenes such as human-computer interaction and rehabilitation assistance.

The application belongs to the field of millimeter wave radar perception and human posture recognition, and discloses a non-contact personnel identity and posture recognition method based on motion instability compensation, comprising the following steps: step 1, static background points are adaptively screened, the radar self-motion speed is estimated, and the point cloud coordinates and the micro-Doppler spectrum are compensated; step 2, a GesAuthNet double-branch deep neural network is built to obtain shape parameters and posture parameters; step 3, an identity authenticator is constructed in combination with the shape parameters to authenticate the legitimacy of the user, and a gesture classifier is constructed in combination with the posture parameters to identify the gesture of the user. The application has the advantages of motion instability compensation capability, robust gesture feature extraction capability, high-precision identity authentication and gesture recognition, and strong stability and robustness.

The application provides a gesture recognition method based on a head-mounted display device, comprising: inputting an image into a hand key point detection model, the hand key point detection model comprising a plurality of stacked feature extraction modules, sequentially adopting the plurality of stacked feature extraction modules to perform feature extraction on the input image to obtain a feature map of a preset size; expanding the feature map of the preset size into a one-dimensional vector through a dimension expansion layer, and adopting a fully connected layer to perform regression prediction of hand key points to obtain coordinates of the hand key points; and obtaining a gesture classification result in the image through a combination relationship between the coordinates of the hand key points. The application combines a hand detection model and a hand key point detection model to realize detection of a gesture, since the network structures of the hand detection model and the hand key point detection model are substantially the same, and only differ in a task head, the full network structure of the gesture detection of the application has small calculation amount and low power consumption under the condition of meeting detection accuracy.

The invention discloses a multi-finger gesture recognition method and system based on frequency modulation continuous waves and an ultrasonic transducer, and belongs to the technical field of man-machine interaction. According to the method, gesture reflection echo signals are collected through a pMUT array, and a dual-channel time-distance Doppler sequence feature map and a time-dual-channel phase difference sequence feature map are obtained through processing such as noise suppression, feature extraction and feature enhancement; and inputting the fused three-channel feature map sequence into a double-flow collaborative attention mechanism-three-dimensional convolutional neural network-long and short-term memory network neural network model to realize multi-finger micro-gesture classification and recognition, and realizing full-process automation based on threshold triggering. On the basis, the invention further provides a pMUT multi-finger micro gesture recognition system. The method has the advantages of being low in operation power consumption, high in environment adaptability, high in resolution precision and the like, and is suitable for the fields of virtual reality, vehicle-mounted control, mechanical arm control and the like.

The invention discloses a hand shape recovery model training method, a hand shape recovery method, an electronic device and a medium, and relates to the technical field of artificial intelligence, the method comprises the steps that a pre-constructed training data set is acquired, and the training data set comprises a category annotation data set, an unannotated data set and a posture annotation data set; performing supervision pre-training on a preset gesture classification model based on the category labeling data set to obtain a pre-trained gesture classification model; constructing a gesture feature model based on the pre-trained gesture classification model, and performing unsupervised pre-training on the gesture feature model based on the unlabeled data set to obtain a pre-trained gesture feature model; and constructing a hand shape recovery model based on the pre-trained gesture feature model, and performing parameter fine tuning training on the hand shape recovery model based on the gesture labeling data set to obtain a trained hand shape recovery model. According to the method, the dependence of hand shape recovery model training on the labeled data is reduced, so that the training cost of the model is reduced.

This invention discloses a five-finger dexterity hand grasping detection method based on soft-mask region representation and multi-task learning. This method decomposes high-dimensional continuous grasping parameters into three sub-tasks: grasping quality prediction, grasping width regression, and grasping gesture classification. It constructs a soft-mask multi-color grasping region representation to generate pixel-level grasping quality, width, and gesture labels. A grasping-oriented channel-space-geometric attention mechanism is designed to construct a lightweight multi-task generative grasping detection network. Taking RGB-D images as input, it outputs grasping quality maps, width maps, and gesture maps in parallel. An adaptive weighted loss function based on effective region constraints is used for training to suppress background interference and dynamically balance multi-task learning. During inference, the grasping center is located by searching for peaks in the quality map, and the corresponding width and gesture are read to achieve single-target or multi-target grasping detection. This invention improves the accuracy, real-time performance, and robustness of grasping detection in multi-object scenes while reducing model complexity.

The invention provides a multi-channel ultrathin flexible stretchable myoelectricity sensor and a preparation method thereof, and relates to the technical field of biological electrodes, the sensor comprises a flexible substrate, a processor and a plurality of sensing units; the sensing units are uniformly arranged on the flexible substrate; each sensing unit is used for acquiring a plurality of electromyographic signals and a plurality of muscle impedance signals within a set time period; the processor is used for performing feature extraction on each electromyographic signal to obtain a plurality of time domain features and a plurality of frequency domain features; the processor is used for performing feature extraction on each muscle impedance signal to obtain a plurality of impedance amplitude features; carrying out splicing fusion to obtain a fusion feature vector; and the processor obtains gesture classification based on the fused feature vector in combination with the classifier. According to the method, the problems of low recognition rate and poor robustness caused by unstable signals, single information dimension and signal asynchronization are solved.

Embodiments of the present application provide a gesture classification method and related equipment, and relate to the field of intelligent control; in the method, hand three-dimensional key points are obtained by using a hand two-dimensional image, the cost of obtaining hand depth information can be reduced without increasing hardware cost; first gesture classification information is determined based on the hand two-dimensional image, and second gesture classification information is determined based on the hand three-dimensional key points; third gesture classification information of the hand two-dimensional image is determined based on the first gesture classification information and the second gesture classification information, the use of rich texture information of the two-dimensional image and spatial information of the three-dimensional key points can be compatible with plane gestures and three-dimensional gestures, and the accuracy of gesture classification can be effectively improved.

The application belongs to the field of biomedical signal processing and human-computer interaction, and discloses a muscle-derived signal gesture recognition method and system based on crown porcupine optimization VMD parameters. The method collects electromyography or muscle magnetic signals during gesture action, and constructs a sample set through sliding window segmentation, data balancing processing and standardization. The crown porcupine optimization algorithm (CPO) is used to adaptively optimize the mode number and penalty factor of variational mode decomposition (VMD), and the gesture classification accuracy is used as the fitness function. Global search and local mining are realized through the four-layer defense of vision, sound, odor and physics of CPO. The intrinsic mode function (IMF) under the optimal parameters is extracted and spliced into a multi-dimensional feature matrix, which is input into a classification model to realize gesture recognition. The application directly optimizes the recognition performance, enhances the feature discriminability, and improves the recognition accuracy and robustness of muscle-derived signals in complex scenes.

The invention discloses a lightweight attitude classification method fusing human body key points and target detection, and relates to the technical field of image processing. The method comprises the following steps: acquiring target image data, and extracting pixel point data of one frame of image in the target image data through a computer vision library; pixel point data of the obtained image is input into a posture estimation model, and the posture estimation model processes the pixel point data and returns the recognized human body key points and the target detection result; performing attitude classification according to a detection result, wherein the attitude can be divided into a static type and a dynamic type; and outputting a final posture form by combining the classification of the static postures and the classification of the dynamic postures. According to the method, a relatively simple machine learning classifier is adopted, and the character postures in the video are classified based on the human body key points and the target detection result which are almost obtained in real time, so that compared with the prior art, the time delay is lower, and the required hardware resources are fewer.

An activity classifier system and method that classifies human activities using 2D skeleton data. The system includes a skeleton preprocessor that transforms the 2D skeleton data into transformed skeleton data, the transformed skeleton data comprising scaled, relative joint positions and relative joint velocities. The system also includes a gesture classifier comprising a first recurrent neural network that receives the transformed skeleton data, and is trained to identify the most probable of a plurality of gestures. The system also has an action classifier comprising a second recurrent neural network that receives information from the first recurrent neural networks and is trained to identify the most probable of a plurality of actions.

The invention discloses a device control method and system and an intelligent ring, and relates to the technical field of wearable devices, the device control method and system are applied to the intelligent ring, the intelligent ring comprises a motion sensor, the intelligent ring is connected with a terminal device, and the method comprises the following steps: obtaining motion data collected by the motion sensor, and performing feature extraction on the motion data to obtain feature data; inputting the feature data into a pre-trained gesture classification model to obtain a gesture classification result; and obtaining a control instruction corresponding to the gesture classification result, and controlling the terminal equipment based on the control instruction. The practicability of the intelligent ring is improved.

The present disclosure provides a gesture recognition method, device, electronic equipment and storage medium, the method comprising: detecting a current video image to obtain at least one hand image containing a hand; then, recognizing each hand image to obtain a gesture classification; thereafter, updating a historical gesture data list according to the gesture classification to obtain a target gesture data list, the target gesture data list comprising a gesture position, a gesture classification and a gesture control right. The present scheme updates the historical gesture data list by gesture classification to track the gesture control right, avoids the problem of misrecognition or inaccurate control right caused by occlusion in the recognition process, and is beneficial to improve the accuracy of gesture recognition.

The invention discloses a remote control manipulator system based on surface electromyogram signals, and relates to the field of remote control robot manipulators. The system comprises an sEMG collecting and processing module used for collecting multi-channel electromyographic signals at the forearms of a human body through a metal dry electrode array, conducting multi-stage amplification and filtering and converting the multi-channel electromyographic signals into digital signals; the wireless transmission and upper computer module is used for transmitting the processed digital signal sequence to an upper computer through a wireless serial port, performing preprocessing and feature extraction on the sEMG signal, and then identifying a predefined gesture category by using a random forest classification model; and the manipulator remote control module is used for remotely transmitting the gesture classification prediction result to the manipulator control unit in real time through the LoRa module so as to remotely control the manipulator to execute the corresponding action. The invention provides a visual and natural manipulator remote control scheme with strong anti-interference capability, and is particularly suitable for the fields of artificial limb control, dangerous environment operation and the like.

The invention discloses an interface interaction control method and device based on machine vision, a terminal and a medium, and relates to the technical field of man-machine interaction, and the method comprises the steps: capturing a hand image of a user in real time through a single RGB camera; detecting a plurality of specified key points of the hand of the user in real time based on a multimedia processing framework hand tracking model and an optimized detection algorithm; constructing a gesture classification model, and identifying a preset type of gesture operation by analyzing the coordinate change of the plurality of specified key points; converting the recognized gesture operation of the preset type into a standard man-machine interface device HID mouse event originally supported by an operating system through a user-defined coordinate mapping algorithm and man-machine interface device protocol conversion logic, and outputting the mouse event to an external display device for display; meanwhile, the recognized gesture operation of the preset type is received, and state feedback and visual prompt are carried out. The man-machine interaction system provided by the invention is low in cost, low in power consumption, plug-and-play and accurate in interaction, and provides convenience for the use of a user.

The invention relates to a direction control system and method. The invention particularly relates to a wearable device comprising a wearable IMU configured to measure a user, a mounting component, and a controller comprising at least one processing core, at least one memory including computer program code; the at least one memory and the computer program code are configured to, with the at least one processing core, cause the device at least to perform: receiving data from the wearable IMU, the data comprising motion information related to a motion of the user, the motion of the user comprising a relative motion between a thumb of the user and a finger of the user, during at least a portion of the motion, a surface of the thumb contacts a surface of the finger, using a gesture classifier and based at least in part on the motion information, classifying the motion of the user as a candidate gesture, the candidate gesture being at least one of: a left directional motion, a right directional motion, an up directional motion, or a down directional motion, and outputting the classified candidate gesture.

The application provides a gesture recognition method, comprising collecting multiple echo intermediate frequency data sent by a microwave sensor; pre-processing the multiple echo intermediate frequency data to obtain multiple target echo intermediate frequency data; performing fast Fourier transform on the multiple target echo intermediate frequency data to calculate multiple frequency data corresponding to the multiple target echo intermediate frequency data and energy values associated with each frequency data; judging whether to trigger gesture recognition according to the energy values; if gesture recognition is triggered, obtaining to-be-recognized data and obtaining a gesture classification result according to the to-be-recognized data. The application performs gesture recognition through a microwave sensor, effectively improves detection efficiency by using the high-resolution characteristics of the microwave sensor, and effectively improves gesture recognition accuracy by performing twice detection on a to-be-detected object according to the judgment of whether to trigger gesture recognition and the judgment of gesture classification recognition.

The invention provides a gesture intention recognition method, system and device and a medium, and belongs to the technical field of computer vision and man-machine interaction, and the method comprises the steps: obtaining a target image containing a hand; extracting coordinate features of the key points of the hand from the image; inputting the coordinate features into a pre-trained classification network to obtain a gesture classification result; and determining a final gesture intention according to a gesture classification result. The method solves the problem that a lightweight model is low in classification performance on edge equipment with limited computing resources.

The invention discloses a gesture recognition control method and device based on virtual reality equipment and a medium, and relates to the technical field of man-machine interaction, and the method comprises the steps: carrying out the dynamic weight fusion of a three-dimensional position of a hand joint point, a motion track feature muscle activation mode and a force intensity feature, and generating a gesture feature vector; inputting a residual neural network to carry out gesture classification and gesture force estimation value calculation, and recognizing gesture information; performing matching analysis on the gesture information and a historical tactile effect library to obtain a gesture feedback mode, and integrating the gesture feedback mode with physical attributes of an object to generate a tactile feedback factor; according to the method, the gesture feedback mode is coupled with the physical attribute of the current interaction object, the tactile feedback factor is generated, dynamic fusion of the tactile parameter, the rigidity coefficient, the damping coefficient and the roughness is achieved, the feedback strength is matched with the material characteristic, and the interaction reality sense is improved.

This invention discloses a method, apparatus, device, and medium for electromyography (EMG) recognition in scenarios with missing source domain data. The method comprises: acquiring source domain EMG signal fragments, inputting them into a TCN feature extractor to extract features, and obtaining fixed-dimensional feature vectors; constructing a source domain gesture recognition model, inputting the feature vectors into a gesture classification head and a domain discriminator head, calculating cross-entropy loss and domain classification loss respectively, and freezing all parameters of the TCN feature extractor and the source domain gesture recognition model after adversarial training until convergence; guiding the user to complete a movement cycle in the target domain through a human-computer interaction interface, acquiring EMG signals and movement labels as calibration data pseudo-labels; constructing a nonlinear transformation module and inserting it into the backend of the frozen TCN feature extractor, generating calibration features aligned with the source domain by performing nonlinear mapping processing on the original feature vectors of the target domain; fine-tuning the nonlinear transformation module using calibration data, and constructing a target domain recognition model to achieve accurate recognition in scenarios with missing source domain data.

The present application belongs to the field of computer vision, and particularly relates to a real-time desktop gesture understanding method, system and device based on time sequence space features, aiming at solving the problem of poor gesture recognition accuracy of existing desktop gesture understanding methods. The method comprises: acquiring a desktop gesture RGB image to be classified and recognized in real time as an input image; converting the input image from an RGB space to an HSV space, segmenting a gesture region in the input image according to a depth value of a pixel point, performing binary processing after segmentation, and obtaining a gesture binary image; fusing gesture binary images in consecutive n frames of input images, and simulating a decay process of consecutive frame gestures by using an exponential decay model to construct a time sequence feature image containing space-time features; and inputting the time sequence feature image containing space-time features into a pre-constructed gesture classification model to obtain a gesture category recognition result corresponding to the input image. The present application improves gesture recognition accuracy.

The application provides a method for recognizing a gesture for air interaction and a terminal device, and relates to the technical field of terminals. The method is applied to a terminal device. When the terminal device is in a bright screen state, the terminal device acquires a first picture at a first frame rate. In response to the first picture including a first gesture, the terminal device acquires a second picture at a second frame rate, the second frame rate is higher than the first frame rate, and the terminal device recognizes a gesture for air interaction based on gesture classification and hand key points of each second picture in multiple second pictures. The higher the frame rate, the more pictures acquired in a unit of time, and the higher the resource memory computing power required by the terminal device to calculate. Therefore, in the above process, whether the terminal device performs the process of acquiring a second picture and recognizing a gesture for air interaction based on the second picture is determined based on whether the first picture includes a first gesture. In this way, the process of the terminal device recognizing a gesture for air interaction can be reduced, thereby reducing power consumption and the cost of computing resources.

The invention relates to the technical field of artificial intelligence, and provides a gesture recognition model establishing method and device and a gesture recognition method and device. The method comprises the following steps: acquiring inertial measurement data and gesture light current data; determining a first training sample corresponding to the inertial measurement data and a second training sample corresponding to the gesture photocurrent data; inputting the first training sample into a generator of a conditional generative adversarial network model to generate a pseudo sample; inputting the second training sample and the pseudo sample into a discriminator of a conditional generative adversarial network model to obtain a classification result; updating parameters of the conditional generative adversarial network model according to the classification result to obtain a trained conditional generative adversarial network model; and fusing the trained conditional generative adversarial network model and a pre-trained gesture classifier to obtain a gesture recognition model. Through the embodiment of the invention, the deployment cost of human body gesture recognition based on the photocurrent data can be greatly reduced on the premise of ensuring the gesture recognition precision.

The invention relates to the technical field of man-machine interaction, in particular to a tactile gesture recognition method based on a pressure sensor, which comprises the following steps: acquiring sensor data generated by tactile gestures through a pressure sensor array at a preset sampling frequency, and implementing time extension and spatial change data enhancement processing on the preprocessed sensor data to obtain a tactile gesture recognition result. Inputting the data subjected to data enhancement processing into a CNN-LSTM-Attention hybrid neural network model for feature extraction and classification, performing weighted fusion on time sequence features by adopting a sequence aggregation module, performing gesture classification by adopting a classification decision-making module, outputting gesture category probability distribution, and identifying a touch gesture type in real time based on a model output result. And an identification result is displayed through a visual interface. Data are collected through the array pressure sensor, spatial-temporal features are extracted in combination with the CNN-LSTM-Attention hybrid network, real-time and high-precision recognition of various touch gestures is achieved, the influence of noise and sensor drift is reduced, and multi-modal gesture recognition is supported.