86 results about "Gesture classification" patented technology

Systems and methods for content processing can include obtaining a gesture input and display data, determining content selected by the gesture input, classifying the gesture, and performing a particular data processing action based on the content selection and the gesture classification. The particular data processing action can vary based on gesture classification. The content selection determination can include determining a gesture mask and then determining the features of the displayed content item that are within the gesture mask.

The invention provides a double-flow circulation attention dynamic gesture recognition method based on motion guidance, and the method comprises the steps: carrying out the motion vector extraction through employing a video coding and decoding tool, quantifying the hand dynamic intensity through combining with a prediction frame residual analysis mechanism, and generating a motion mask; sequentially performing gating fusion and graph convolution calculation on the RGB features and the hand motion contour mask by using a gating network and a graph neural network; and utilizing a dynamic space-time attenuation module and a multi-scale wavelet spectrum-space block to respectively carry out expansion convolution processing and multi-scale frequency domain reconstruction and channel interaction to obtain a gesture classification result. The invention provides a circulation attention mechanism based on a full convolution architecture, and efficient and low-complexity time sequence modeling is realized in combination with a learnable time decay strategy. Meanwhile, a wavelet decomposition reconstruction and multi-scale pooling method is adopted to capture fine-grained hand motion changes, so that the network can effectively model short time sequence dependence features, and the recognition precision of dynamic gestures is further improved.

The embodiment of the invention belongs to the field of computer vision, and relates to a dynamic gesture recognition method, which comprises the following steps: acquiring dynamic gesture data; inputting the dynamic gesture data into a pre-trained dynamic gesture recognition model for recognition processing to obtain a gesture classification result; and identifying a target gesture action based on the gesture classification result. The invention further provides a dynamic gesture recognition device, computer equipment and a storage medium. According to the method, the problems that feature attention of key information of an existing dynamic gesture recognition model is not accurate enough and the generalization ability of the model is limited are solved, and the dynamic gesture classification accuracy is improved.

The invention discloses a gesture recognition method and device and electronic equipment, and belongs to the technical field of gesture recognition. The method comprises the following steps: carrying out hand detection on a picture containing a hand image by adopting a pre-trained hand detection model, and obtaining position information of a hand detection frame corresponding to the picture; determining a hand region in the picture based on the position information of the hand detection frame; adopting a pre-trained hand key point detection model to detect hand key points contained in the picture of the hand region, and obtaining position information of the hand key points; and performing gesture recognition processing on the position information of the hand key points by adopting a pre-trained gesture recognition model to obtain a gesture recognition result matched with the hand image. According to the method, the consecutive hand detection, hand key point detection and gesture classification processes are adopted, so that the accuracy of hand key point detection is improved, the accuracy of gesture recognition is improved, and the robustness of gesture recognition in a complex environment can be improved.

The invention relates to the technical field of computer vision and human-computer interaction, and discloses a self-adaptive hand image recognition method and system based on multi-source feature fusion. The method comprises the following steps: acquiring a real-time video stream image from a camera, carrying out various filtering processing on the real-time video stream image, and respectively generating an illumination enhanced image, a background subtraction mask, a skin color area mask and a dynamic motion mask; dynamically adjusting the fusion weight of each filtering processing result according to the scene features of the real-time video stream image, and generating comprehensive filtering output through weighted fusion; performing multi-stage hand segmentation processing on the comprehensive filtering output to obtain an initial hand region mask, performing time sequence smoothing processing, and extracting a maximum connected domain as a final hand region mask; and inputting the final hand region mask into a convolutional neural network for gesture recognition, outputting a prediction probability of each type of gestures, and displaying a gesture classification result in real time. According to the method, the adaptability of hand detection to strong interference backgrounds and variable environments is enhanced.

The present application relates to the technical field of head-mounted display devices, and provides a control method, a model training method, an apparatus, a device, and a storage medium. The control method comprises: acquiring an image to be recognized, said image comprising at least part of a hand; extracting a hand feature vector of said image; on the basis of a gesture recognition network, performing key point detection on the hand feature vector to obtain key point positions of a plurality of preset hand key points, and on the basis of the hand feature vector, determining a gesture category corresponding to said image; and on the basis of the key point position of at least one preset hand key point and the gesture category, controlling a head-mounted display device to execute a preset task. The gesture classification of the present application does not rely on the result of key point detection, the present application is applicable to a head-mounted display device with low computing power, and on the basis of the key point positions and the gesture category, the head-mounted display device is controlled to execute a task, enriching human-computer interaction modes, improving the interaction experience of users and devices.

Systems and methods for content processing can include obtaining a gesture input and display data, determining content selected by the gesture input, classifying the gesture, and performing a particular data processing action based on the content selection and the gesture classification. The particular data processing action can vary based on gesture classification. The content selection determination can include determining a gesture mask and then determining the features of the displayed content item that are within the gesture mask.

The invention discloses a gesture recognition method based on entropy features and an improved PSO-SVM. The gesture recognition method comprises the following steps that S1, surface electromyogram signals of multiple gestures are collected and preprocessed; s2, performing multi-scale decomposition on the surface electromyogram signals by adopting a variational mode decomposition method, and extracting entropy features; s3, performing dimension reduction processing on the entropy features by using correlation analysis, a random forest and a Lasso method to obtain dimension-reduced feature data; and S4, classifying the dimension reduction feature data by adopting a gesture classification model to realize gesture recognition. According to the method, the entropy features of the surface electromyogram signals are extracted through the variational mode decomposition method, the parameters of the support vector machine are optimized in combination with the particle swarm optimization algorithm, the gesture classification model is constructed, the influence of a traditional time domain and frequency domain feature extraction mode on the robustness of the gesture classification model is effectively solved, and the classification precision and robustness are improved.

The invention discloses a multi-channel gesture recognition method and device, and belongs to the field of rehabilitation. N-channel electromyographic signals during hand action and real-time pressure signals between finger pulps and grabbed objects are collected; the N-channel electromyographic signals are preprocessed according to the real-time pressure signals; the preprocessed N-channel electromyographic signals are integrated into a signal matrix; time domain, frequency domain, time-frequency domain and space correlation features are sequentially extracted from the signal matrix, and all the features of each time frame are spliced according to the time sequence to form a high-dimensional feature matrix; the high-dimensional feature matrix is input into a dynamic space-time diagram convolutional network model, the dynamic space-time diagram convolutional network model carries out gesture and force recognition, gesture classification probability distribution and force classification probability distribution are formed, and a result is output, and through the steps, complex gestures and gestures with different forces can be accurately classified and recognized.

The invention provides a dynamic gesture recognition method and device, electronic equipment, a chip and a medium, and relates to the technical field of man-machine interaction. The method comprises the following steps: carrying out key point identification on an image in a video frame collected by a binocular camera, and respectively determining a first three-dimensional key point and a second three-dimensional key point; identifying the gesture category of the target gesture in the video frame as a first gesture category by using a dynamic gesture classification model; based on the corresponding relation between the reference waveform feature and the gesture category, using a signal recognition algorithm to analyze the waveform feature formed by the first stereo key point and / or the second stereo key point, and recognizing a second gesture category corresponding to the target gesture in the video frame; and checking whether the first gesture category is consistent with the second gesture category to determine a target gesture category. Through the technical scheme provided by the invention, the problems of low accuracy and low stability of dynamic gesture recognition are solved, the accuracy and the stability of gesture recognition are improved, and the user experience of gesture interaction is improved.

The invention relates to the technical field of head-mounted display equipment, and provides a control method, a model training method and device, equipment and a storage medium, and the control method comprises the steps: obtaining a to-be-recognized image which comprises at least a part of a hand; extracting a hand feature vector of the to-be-recognized image; performing key point detection on the hand feature vector based on a gesture recognition network to obtain key point positions of a plurality of preset hand key points, and determining a gesture category corresponding to the to-be-recognized image according to the hand feature vector; and controlling the head-mounted display device to execute a preset task according to the key point position of the at least one preset hand key point and the gesture category. The gesture classification does not depend on the key point detection result, the method is suitable for the head-mounted display device with low computing power, the head-mounted display device is controlled to execute tasks according to the key point position and the gesture category, the man-machine interaction mode is enriched, and the interaction experience of a user and the device is improved.

The invention discloses a virtual interaction hand mirror image rehabilitation training method and system based on myoelectricity control, relates to the technical field of rehabilitation machine learning, and aims to solve the problems of low recognition precision and poor tactile feedback in virtual interaction rehabilitation training in the prior art. Performing feature extraction on the real-time electromyographic signal data of the healthy side hand of the patient to obtain time domain features and frequency domain features; taking the time domain feature and the frequency domain feature as input, and outputting based on a convolutional neural network model to obtain a gesture classification result; according to the gesture classification result, the hand rehabilitation robot is used for carrying out mirror image training on the hand on the affected side, and interaction between a virtual game and real actions of the patient is achieved through game software. The motion intention of the patient is predicted in advance by collecting the electromyographic signals, the patient is better guided to conduct mirror image training, interaction between the actual motion of the patient and the virtual game is achieved, and the pleasure and enthusiasm of the patient in rehabilitation training can be increased.

Systems and methods for content processing can include obtaining a gesture input and display data, determining content selected by the gesture input, classifying the gesture, and performing a particular data processing action based on the content selection and the gesture classification. The particular data processing action can vary based on gesture classification. The content selection determination can include determining a gesture mask and then determining the features of the displayed content item that are within the gesture mask.

An activity classifier system and method that classifies human activities using 2D skeleton data. The system includes a skeleton preprocessor that transforms the 2D skeleton data into transformed skeleton data, the transformed skeleton data comprising scaled, relative joint positions and relative joint velocities. The system also includes a gesture classifier comprising a first recurrent neural network that receives the transformed skeleton data, and is trained to identify the most probable of a plurality of gestures. The system also has an action classifier comprising a second recurrent neural network that receives information from the first recurrent neural networks and is trained to identify the most probable of a plurality of actions.

Methods and apparatuses (devices, methods, etc.) for determining gestures, e.g., hand gestures, from a wrist and / or forearm worn array of magnetometers. These methods and apparatuses may use a trained machine-learning agent to identify gestures from magnetic signals that have been preprocessed to optimize gesture detection. The trained machine learning agent may apply a hierarchical classification that first identifies a first category of gestures from a second category of gestures and then classifies between the individual gestures within each category. The methods and apparatuses described herein may be of particular use with acoustically driven ferromagnetic resonance (ADFMR) sensors.

The invention discloses a control method and device of an electronic rearview mirror system and the electronic rearview mirror system. The method comprises the steps that a CMS controller obtains a user video stream collected by a first camera; analyzing and processing the user video stream, and determining whether a gesture image exists in the user video stream; if yes, triggering gesture classification on the gesture image in the user video stream to obtain a recognition gesture; determining an operation instruction corresponding to the recognition gesture based on a preset mapping relationship between the recognition gesture and the operation instruction; the operation instruction is executed to control CMS display content, the display content is obtained based on a view field image collected by a second camera, and the second camera is arranged in a rearview mirror outside the cabin. According to the scheme, the gesture action of the driver is monitored through the DMS, the gesture video stream is transmitted to the CMS monitor in real time, intelligent interaction between display of the CMS monitor and the driver is achieved, and the monitor can be rapidly and conveniently adjusted by adjusting the display of the visible field of view of the monitor through gestures.

The invention relates to the technical field of virtual classrooms, and discloses a real-time collaborative virtual classroom interaction method, which comprises the following steps of: constructing a classroom gesture operation comparison table according to preset virtual classroom gestures; capturing an infrared image of a hand of a user and joint point coordinate data in real time, and performing preprocessing; performing feature extraction on the preprocessed infrared image to realize gesture classification to obtain gesture description; searching in a classroom gesture operation comparison table according to the gesture description to determine whether gesture motion judgment is started or not, and determining a virtual object corresponding to the gesture; detecting an operation request user of the same virtual object in the same time interval, and executing a preset conflict execution mechanism; and executing the corresponding virtual operation result so as to map the virtual operation result into virtual classroom operation. According to the method, the gesture comparison table is constructed, data collection and recognition and multi-user conflict processing are optimized, accurate mapping from gestures to virtual operation is achieved, interaction accuracy and cooperation fluency are improved, and immersive experience of a virtual classroom is enhanced.

The invention provides a gesture recognition method based on an optical fiber sensing technology and deep learning, and relates to the technical field of optical fiber sensing. According to the gesture recognition method, gesture information is captured through speckle image changes of the multimode optical fiber fixed on a glove under different gestures by utilizing the deformation characteristic of the multimode optical fiber, and high-precision gesture classification is carried out in combination with a convolutional neural network (CNN) and a self-attention mechanism. The specific implementation of the method comprises the steps of layout and fixation of multimode optical fibers, speckle image acquisition, data preprocessing, neural network model design and training and the like. Wherein a 635nm semiconductor laser is used as a light source, a CCD camera is used for collecting speckle images in real time, a Python script is used for carrying out optimization processing and data enhancement on the images, a final deep learning model is composed of a feature extraction layer, a global feature modeling layer and a classification layer, and high-precision recognition of gestures is achieved.

The embodiment of the invention discloses a gesture recognition method and electronic equipment. The method comprises the following steps: performing image acquisition on a user gesture through an image acquisition device to obtain a continuous video sequence; performing gesture region extraction on each image frame in the continuous video sequence to obtain a gesture image, and performing feature extraction on the gesture image to obtain spatial stream features; performing optical flow calculation on two continuous frames of images to obtain optical flow images, and performing feature extraction on the optical flow images to obtain time flow features; and fusing the spatial flow features and the time flow features to obtain fused features, and performing gesture classification on the fused features based on a classifier to determine a gesture recognition result. According to the scheme, the space features and the time features can be subjected to cross-modal correlation fusion, so that the two features are complementary in gesture recognition, the motion features can know that the apparent features pay attention to a dynamic area, the border features can correct misjudgment of the operation features, and the accuracy of gesture recognition is improved.

The present invention discloses a method and device applicable to underwater pose recognition, relating to the technical field of underwater detection, including: S1, input an image, and extract visual features through a feature extraction network; S2, construct a position embedding, add by position, and send the result to the transformer encoder E; S3, adopt a pre-trained model CLIP to extract image features; S4, after decoding the output result of E, combine it with the features extracted by CLIP as the true features and input them into the structure of the GAN. The method and device applicable to underwater pose recognition combine the strong representation learning ability of the Transform structure as a visual feature extractor, and then use the generative adversarial network to synthesize visual features of unseen gestures, enabling us to train a gesture classifier using data from both visible and unseen data, overcoming the weaknesses of supervised learning.

In an embodiment, a method includes: obtaining one or more positional time spectrograms of a radar measurement of a scene comprising an object; and based on the one or more positional time spectrograms and based on a feature embedding of a variational auto-encoder neural network, predicting a gesture class of a gesture performed by the object.

The invention relates to the technical field of electromyographic signal processing, in particular to an electromyographic acquisition signal processing method and system for gesture classification. The method comprises the following steps: forming a signal label and an electromyographic signal sample cluster based on a gesture type, electromyographic signal acquisition duration and gesture recognition accuracy; calibrating a cross-cluster error value of the electromyographic signal features; mapping the cross-cluster error value to a two-dimensional coordinate system and adding a point feature value; and generating a to-be-processed sample set through the dual-scale data screening window, judging a gesture type membership degree, and executing visual display. According to the method, the classification precision problem caused by user action habit differences is solved through intelligent data analysis, the accuracy and adaptability of gesture classification are improved, and the method is suitable for scenes such as human-computer interaction and rehabilitation assistance.

The present invention proposes a two-stream cyclic attention dynamic gesture recognition method based on motion guidance. The method includes: using a video codec tool to extract motion vectors, and combining a prediction frame residual analysis mechanism to quantify the dynamic intensity of the hand and generate a motion mask; using a gated network and a graph neural network for the RGB features and the hand motion contour mask to perform gated fusion and graph convolution calculations in sequence; using a dynamic spatio-temporal decay module and a multi-scale wavelet spectrum-spatial block to perform dilated convolution processing, as well as multi-scale frequency domain reconstruction and channel interaction respectively to obtain a gesture classification result. The present invention proposes a cyclic attention mechanism based on a fully convolutional architecture, combined with a learnable time decay strategy, to achieve efficient and low-complexity temporal modeling. At the same time, a wavelet decomposition and reconstruction and multi-scale pooling method are adopted to capture fine-grained hand motion changes, enabling the network to effectively model short-term temporal dependence features and further improving the recognition accuracy of dynamic gestures.

The application belongs to the field of millimeter wave radar perception and human posture recognition, and discloses a non-contact personnel identity and posture recognition method based on motion instability compensation, comprising the following steps: step 1, static background points are adaptively screened, the radar self-motion speed is estimated, and the point cloud coordinates and the micro-Doppler spectrum are compensated; step 2, a GesAuthNet double-branch deep neural network is built to obtain shape parameters and posture parameters; step 3, an identity authenticator is constructed in combination with the shape parameters to authenticate the legitimacy of the user, and a gesture classifier is constructed in combination with the posture parameters to identify the gesture of the user. The application has the advantages of motion instability compensation capability, robust gesture feature extraction capability, high-precision identity authentication and gesture recognition, and strong stability and robustness.

The application provides a gesture recognition method based on a head-mounted display device, comprising: inputting an image into a hand key point detection model, the hand key point detection model comprising a plurality of stacked feature extraction modules, sequentially adopting the plurality of stacked feature extraction modules to perform feature extraction on the input image to obtain a feature map of a preset size; expanding the feature map of the preset size into a one-dimensional vector through a dimension expansion layer, and adopting a fully connected layer to perform regression prediction of hand key points to obtain coordinates of the hand key points; and obtaining a gesture classification result in the image through a combination relationship between the coordinates of the hand key points. The application combines a hand detection model and a hand key point detection model to realize detection of a gesture, since the network structures of the hand detection model and the hand key point detection model are substantially the same, and only differ in a task head, the full network structure of the gesture detection of the application has small calculation amount and low power consumption under the condition of meeting detection accuracy.

The present invention discloses a wearable gesture interaction system and method based on multimodal flexible sensing. The wearable gesture interaction system includes: two modal flexible sensors, namely a flexible strain sensor and a flexible acceleration sensor, for capturing gesture simulation signals indicating finger bending state, three-dimensional hand posture, and hand swiping; a data acquisition module for converting the gesture simulation signals into digital gesture signals and outputting them in real time via a wireless communication module; and a host computer for receiving the digital gesture signals, performing gesture activity segmentation and preprocessing to obtain a Grammar angle field map of the gesture signals. A deep learning model based on a self-attention mechanism is then used to extract and fuse multimodal features to achieve gesture classification for gesture human-computer interaction. The present invention uses multimodal flexible sensors combined with deep learning to improve the accuracy of gesture recognition. Furthermore, the system is comfortable to wear and relatively low in cost, effectively meeting user requirements.

The invention discloses a multi-finger gesture recognition method and system based on frequency modulation continuous waves and an ultrasonic transducer, and belongs to the technical field of man-machine interaction. According to the method, gesture reflection echo signals are collected through a pMUT array, and a dual-channel time-distance Doppler sequence feature map and a time-dual-channel phase difference sequence feature map are obtained through processing such as noise suppression, feature extraction and feature enhancement; and inputting the fused three-channel feature map sequence into a double-flow collaborative attention mechanism-three-dimensional convolutional neural network-long and short-term memory network neural network model to realize multi-finger micro-gesture classification and recognition, and realizing full-process automation based on threshold triggering. On the basis, the invention further provides a pMUT multi-finger micro gesture recognition system. The method has the advantages of being low in operation power consumption, high in environment adaptability, high in resolution precision and the like, and is suitable for the fields of virtual reality, vehicle-mounted control, mechanical arm control and the like.

The invention discloses a hand shape recovery model training method, a hand shape recovery method, an electronic device and a medium, and relates to the technical field of artificial intelligence, the method comprises the steps that a pre-constructed training data set is acquired, and the training data set comprises a category annotation data set, an unannotated data set and a posture annotation data set; performing supervision pre-training on a preset gesture classification model based on the category labeling data set to obtain a pre-trained gesture classification model; constructing a gesture feature model based on the pre-trained gesture classification model, and performing unsupervised pre-training on the gesture feature model based on the unlabeled data set to obtain a pre-trained gesture feature model; and constructing a hand shape recovery model based on the pre-trained gesture feature model, and performing parameter fine tuning training on the hand shape recovery model based on the gesture labeling data set to obtain a trained hand shape recovery model. According to the method, the dependence of hand shape recovery model training on the labeled data is reduced, so that the training cost of the model is reduced.

This invention discloses a five-finger dexterity hand grasping detection method based on soft-mask region representation and multi-task learning. This method decomposes high-dimensional continuous grasping parameters into three sub-tasks: grasping quality prediction, grasping width regression, and grasping gesture classification. It constructs a soft-mask multi-color grasping region representation to generate pixel-level grasping quality, width, and gesture labels. A grasping-oriented channel-space-geometric attention mechanism is designed to construct a lightweight multi-task generative grasping detection network. Taking RGB-D images as input, it outputs grasping quality maps, width maps, and gesture maps in parallel. An adaptive weighted loss function based on effective region constraints is used for training to suppress background interference and dynamically balance multi-task learning. During inference, the grasping center is located by searching for peaks in the quality map, and the corresponding width and gesture are read to achieve single-target or multi-target grasping detection. This invention improves the accuracy, real-time performance, and robustness of grasping detection in multi-object scenes while reducing model complexity.