102 results about "Gesture classification" patented technology

The invention belongs to the field of gesture recognition, and provides a multi-head attention and spatial-temporal feature fusion-based gesture recognition method and system, and the method comprises the steps: building a neural network model structure through a deep learning frame, importing a gesture sequence of hand articulation points containing time sequence features, and carrying out the recognition of the gesture sequence; and performing feature extraction, position coding and attention calculation, feature fusion and feature mapping on the gesture sequence data by using a proposed gesture recognition algorithm, and outputting a gesture classification result with higher accuracy. A new network topology structure is adopted, and a local and global multi-head attention module is utilized to capture a complex relationship in space and time in two dimensions of time and space, so that the problem that a current model is insufficient in capturing short-term and long-term dependency relationships when coping with complex time sequence data is solved; and the fusion algorithm is more flexible and efficient when processing different types of gesture sequence data.

Systems and methods for content processing can include obtaining a gesture input and display data, determining content selected by the gesture input, classifying the gesture, and performing a particular data processing action based on the content selection and the gesture classification. The particular data processing action can vary based on gesture classification. The content selection determination can include determining a gesture mask and then determining the features of the displayed content item that are within the gesture mask.

The invention provides a double-flow circulation attention dynamic gesture recognition method based on motion guidance, and the method comprises the steps: carrying out the motion vector extraction through employing a video coding and decoding tool, quantifying the hand dynamic intensity through combining with a prediction frame residual analysis mechanism, and generating a motion mask; sequentially performing gating fusion and graph convolution calculation on the RGB features and the hand motion contour mask by using a gating network and a graph neural network; and utilizing a dynamic space-time attenuation module and a multi-scale wavelet spectrum-space block to respectively carry out expansion convolution processing and multi-scale frequency domain reconstruction and channel interaction to obtain a gesture classification result. The invention provides a circulation attention mechanism based on a full convolution architecture, and efficient and low-complexity time sequence modeling is realized in combination with a learnable time decay strategy. Meanwhile, a wavelet decomposition reconstruction and multi-scale pooling method is adopted to capture fine-grained hand motion changes, so that the network can effectively model short time sequence dependence features, and the recognition precision of dynamic gestures is further improved.

The invention provides a dynamic gesture recognition method based on multi-modal fusion. The method comprises the following steps: firstly, acquiring a dynamic gesture data set as a training sample, including video streams of two modalities of an RGB image and a depth image; preprocessing the training sample; constructing a multi-modal gesture classification and recognition model; a dynamic gradient modulation method optimization algorithm is adopted, and the preprocessed training samples are input into the constructed multi-modal gesture classification recognition model for training; and obtaining to-be-recognized dynamic gesture data, preprocessing the to-be-recognized dynamic gesture data, and inputting the preprocessed to-be-recognized dynamic gesture data into the trained multi-modal gesture classification recognition model to obtain a classification recognition result. According to the dynamic gesture recognition method based on multi-modal fusion provided by the invention, heterogeneity between modals is effectively reduced by balancing optimization speeds of different modals, complementary information of an RGB image and a depth image can be fully utilized, rich feature information can be obtained, and the performance of a multi-modal dynamic gesture recognition model is improved.

The invention discloses a gesture recognition method and device based on YOLOv5-FGS and Mediape and a medium. The method comprises the steps that a hand detection data set is constructed, and an improved YOLOv5-FGS hand detection model is obtained through training; performing hand detection on the gesture classification data set by using a YOLOv5-FGS hand detection model, performing hand key point detection through Mediape, training a naive Bayes classifier in combination with hand key point information output by the Mediape, and generating a gesture classification model; according to a video image to be detected, single-frame detection and hand position positioning are firstly carried out through the YOLOv5-FGS hand detection model, then hand key point detection is carried out through Mediape, hand key point features obtained through processing are detected through the gesture classification model, and a final gesture target positioning result and a final gesture target classification result are output.

The embodiment of the invention belongs to the field of computer vision, and relates to a dynamic gesture recognition method, which comprises the following steps: acquiring dynamic gesture data; inputting the dynamic gesture data into a pre-trained dynamic gesture recognition model for recognition processing to obtain a gesture classification result; and identifying a target gesture action based on the gesture classification result. The invention further provides a dynamic gesture recognition device, computer equipment and a storage medium. According to the method, the problems that feature attention of key information of an existing dynamic gesture recognition model is not accurate enough and the generalization ability of the model is limited are solved, and the dynamic gesture classification accuracy is improved.

The invention discloses a gesture recognition method and device and electronic equipment, and belongs to the technical field of gesture recognition. The method comprises the following steps: carrying out hand detection on a picture containing a hand image by adopting a pre-trained hand detection model, and obtaining position information of a hand detection frame corresponding to the picture; determining a hand region in the picture based on the position information of the hand detection frame; adopting a pre-trained hand key point detection model to detect hand key points contained in the picture of the hand region, and obtaining position information of the hand key points; and performing gesture recognition processing on the position information of the hand key points by adopting a pre-trained gesture recognition model to obtain a gesture recognition result matched with the hand image. According to the method, the consecutive hand detection, hand key point detection and gesture classification processes are adopted, so that the accuracy of hand key point detection is improved, the accuracy of gesture recognition is improved, and the robustness of gesture recognition in a complex environment can be improved.

The invention relates to the technical field of computer vision and human-computer interaction, and discloses a self-adaptive hand image recognition method and system based on multi-source feature fusion. The method comprises the following steps: acquiring a real-time video stream image from a camera, carrying out various filtering processing on the real-time video stream image, and respectively generating an illumination enhanced image, a background subtraction mask, a skin color area mask and a dynamic motion mask; dynamically adjusting the fusion weight of each filtering processing result according to the scene features of the real-time video stream image, and generating comprehensive filtering output through weighted fusion; performing multi-stage hand segmentation processing on the comprehensive filtering output to obtain an initial hand region mask, performing time sequence smoothing processing, and extracting a maximum connected domain as a final hand region mask; and inputting the final hand region mask into a convolutional neural network for gesture recognition, outputting a prediction probability of each type of gestures, and displaying a gesture classification result in real time. According to the method, the adaptability of hand detection to strong interference backgrounds and variable environments is enhanced.

The present invention proposes a dynamic gesture recognition method based on multi-view three-dimensional skeleton information fusion. It includes: using a sliding window-based detection method to segment the original dynamic multi-gesture sequence to obtain multiple single gesture sequences; performing a three-dimensional space coordinate transformation on each single gesture sequence to obtain the corresponding multi-view three-dimensional skeleton information; encoding each multi-view three-dimensional skeleton information to obtain a single-view total skeleton map corresponding to multiple perspectives; inputting the single-view total skeleton map of each perspective into the corresponding branch convolutional neural network for feature extraction, and then inputting it into the aggregation network based on the perspective attention mechanism, and then inputting it into the flattening layer and the fully connected layer in turn, and the fully connected layer outputs the single gesture classification result. The present invention can solve the problems of insufficient utilization of spatial information, difficult recognition of complex gestures, poor robustness, etc. in the traditional gesture recognition method using a single perspective, and greatly improves the recognition accuracy.

The present application relates to the technical field of head-mounted display devices, and provides a control method, a model training method, an apparatus, a device, and a storage medium. The control method comprises: acquiring an image to be recognized, said image comprising at least part of a hand; extracting a hand feature vector of said image; on the basis of a gesture recognition network, performing key point detection on the hand feature vector to obtain key point positions of a plurality of preset hand key points, and on the basis of the hand feature vector, determining a gesture category corresponding to said image; and on the basis of the key point position of at least one preset hand key point and the gesture category, controlling a head-mounted display device to execute a preset task. The gesture classification of the present application does not rely on the result of key point detection, the present application is applicable to a head-mounted display device with low computing power, and on the basis of the key point positions and the gesture category, the head-mounted display device is controlled to execute a task, enriching human-computer interaction modes, improving the interaction experience of users and devices.

Systems and methods for content processing can include obtaining a gesture input and display data, determining content selected by the gesture input, classifying the gesture, and performing a particular data processing action based on the content selection and the gesture classification. The particular data processing action can vary based on gesture classification. The content selection determination can include determining a gesture mask and then determining the features of the displayed content item that are within the gesture mask.

The invention discloses a gesture recognition method based on entropy features and an improved PSO-SVM. The gesture recognition method comprises the following steps that S1, surface electromyogram signals of multiple gestures are collected and preprocessed; s2, performing multi-scale decomposition on the surface electromyogram signals by adopting a variational mode decomposition method, and extracting entropy features; s3, performing dimension reduction processing on the entropy features by using correlation analysis, a random forest and a Lasso method to obtain dimension-reduced feature data; and S4, classifying the dimension reduction feature data by adopting a gesture classification model to realize gesture recognition. According to the method, the entropy features of the surface electromyogram signals are extracted through the variational mode decomposition method, the parameters of the support vector machine are optimized in combination with the particle swarm optimization algorithm, the gesture classification model is constructed, the influence of a traditional time domain and frequency domain feature extraction mode on the robustness of the gesture classification model is effectively solved, and the classification precision and robustness are improved.

The invention discloses a multi-channel gesture recognition method and device, and belongs to the field of rehabilitation. N-channel electromyographic signals during hand action and real-time pressure signals between finger pulps and grabbed objects are collected; the N-channel electromyographic signals are preprocessed according to the real-time pressure signals; the preprocessed N-channel electromyographic signals are integrated into a signal matrix; time domain, frequency domain, time-frequency domain and space correlation features are sequentially extracted from the signal matrix, and all the features of each time frame are spliced according to the time sequence to form a high-dimensional feature matrix; the high-dimensional feature matrix is input into a dynamic space-time diagram convolutional network model, the dynamic space-time diagram convolutional network model carries out gesture and force recognition, gesture classification probability distribution and force classification probability distribution are formed, and a result is output, and through the steps, complex gestures and gestures with different forces can be accurately classified and recognized.

The invention provides a dynamic gesture recognition method and device, electronic equipment, a chip and a medium, and relates to the technical field of man-machine interaction. The method comprises the following steps: carrying out key point identification on an image in a video frame collected by a binocular camera, and respectively determining a first three-dimensional key point and a second three-dimensional key point; identifying the gesture category of the target gesture in the video frame as a first gesture category by using a dynamic gesture classification model; based on the corresponding relation between the reference waveform feature and the gesture category, using a signal recognition algorithm to analyze the waveform feature formed by the first stereo key point and / or the second stereo key point, and recognizing a second gesture category corresponding to the target gesture in the video frame; and checking whether the first gesture category is consistent with the second gesture category to determine a target gesture category. Through the technical scheme provided by the invention, the problems of low accuracy and low stability of dynamic gesture recognition are solved, the accuracy and the stability of gesture recognition are improved, and the user experience of gesture interaction is improved.

The invention relates to the technical field of head-mounted display equipment, and provides a control method, a model training method and device, equipment and a storage medium, and the control method comprises the steps: obtaining a to-be-recognized image which comprises at least a part of a hand; extracting a hand feature vector of the to-be-recognized image; performing key point detection on the hand feature vector based on a gesture recognition network to obtain key point positions of a plurality of preset hand key points, and determining a gesture category corresponding to the to-be-recognized image according to the hand feature vector; and controlling the head-mounted display device to execute a preset task according to the key point position of the at least one preset hand key point and the gesture category. The gesture classification does not depend on the key point detection result, the method is suitable for the head-mounted display device with low computing power, the head-mounted display device is controlled to execute tasks according to the key point position and the gesture category, the man-machine interaction mode is enriched, and the interaction experience of a user and the device is improved.

The invention discloses a virtual interaction hand mirror image rehabilitation training method and system based on myoelectricity control, relates to the technical field of rehabilitation machine learning, and aims to solve the problems of low recognition precision and poor tactile feedback in virtual interaction rehabilitation training in the prior art. Performing feature extraction on the real-time electromyographic signal data of the healthy side hand of the patient to obtain time domain features and frequency domain features; taking the time domain feature and the frequency domain feature as input, and outputting based on a convolutional neural network model to obtain a gesture classification result; according to the gesture classification result, the hand rehabilitation robot is used for carrying out mirror image training on the hand on the affected side, and interaction between a virtual game and real actions of the patient is achieved through game software. The motion intention of the patient is predicted in advance by collecting the electromyographic signals, the patient is better guided to conduct mirror image training, interaction between the actual motion of the patient and the virtual game is achieved, and the pleasure and enthusiasm of the patient in rehabilitation training can be increased.

Systems and methods for content processing can include obtaining a gesture input and display data, determining content selected by the gesture input, classifying the gesture, and performing a particular data processing action based on the content selection and the gesture classification. The particular data processing action can vary based on gesture classification. The content selection determination can include determining a gesture mask and then determining the features of the displayed content item that are within the gesture mask.

An activity classifier system and method that classifies human activities using 2D skeleton data. The system includes a skeleton preprocessor that transforms the 2D skeleton data into transformed skeleton data, the transformed skeleton data comprising scaled, relative joint positions and relative joint velocities. The system also includes a gesture classifier comprising a first recurrent neural network that receives the transformed skeleton data, and is trained to identify the most probable of a plurality of gestures. The system also has an action classifier comprising a second recurrent neural network that receives information from the first recurrent neural networks and is trained to identify the most probable of a plurality of actions.

Methods and apparatuses (devices, methods, etc.) for determining gestures, e.g., hand gestures, from a wrist and / or forearm worn array of magnetometers. These methods and apparatuses may use a trained machine-learning agent to identify gestures from magnetic signals that have been preprocessed to optimize gesture detection. The trained machine learning agent may apply a hierarchical classification that first identifies a first category of gestures from a second category of gestures and then classifies between the individual gestures within each category. The methods and apparatuses described herein may be of particular use with acoustically driven ferromagnetic resonance (ADFMR) sensors.

The invention discloses a control method and device of an electronic rearview mirror system and the electronic rearview mirror system. The method comprises the steps that a CMS controller obtains a user video stream collected by a first camera; analyzing and processing the user video stream, and determining whether a gesture image exists in the user video stream; if yes, triggering gesture classification on the gesture image in the user video stream to obtain a recognition gesture; determining an operation instruction corresponding to the recognition gesture based on a preset mapping relationship between the recognition gesture and the operation instruction; the operation instruction is executed to control CMS display content, the display content is obtained based on a view field image collected by a second camera, and the second camera is arranged in a rearview mirror outside the cabin. According to the scheme, the gesture action of the driver is monitored through the DMS, the gesture video stream is transmitted to the CMS monitor in real time, intelligent interaction between display of the CMS monitor and the driver is achieved, and the monitor can be rapidly and conveniently adjusted by adjusting the display of the visible field of view of the monitor through gestures.

Method and apparatus for training a classification model and using the trained classification model to recognize a gesture performed by a user. An apparatus includes a processor programmed to: receive a first plurality of neuromuscular signals from a user via a plurality of neuromuscular sensors when the user performs a first single action of a gesture; train a classification model based on the first plurality of neuromuscular signals, the training comprising: deriving values ​​from the first plurality of neuromuscular signals, the values ​​indicating distinctive features of the gesture, the distinctive features comprising at least one feature that varies linearly with force applied during performance of the gesture; and generating a first classification representation of the gesture in the classification model based on the values; and determining that the user performed a second single action of the gesture based on the trained classification model and the second plurality of neuromuscular signals.

The invention relates to the technical field of virtual classrooms, and discloses a real-time collaborative virtual classroom interaction method, which comprises the following steps of: constructing a classroom gesture operation comparison table according to preset virtual classroom gestures; capturing an infrared image of a hand of a user and joint point coordinate data in real time, and performing preprocessing; performing feature extraction on the preprocessed infrared image to realize gesture classification to obtain gesture description; searching in a classroom gesture operation comparison table according to the gesture description to determine whether gesture motion judgment is started or not, and determining a virtual object corresponding to the gesture; detecting an operation request user of the same virtual object in the same time interval, and executing a preset conflict execution mechanism; and executing the corresponding virtual operation result so as to map the virtual operation result into virtual classroom operation. According to the method, the gesture comparison table is constructed, data collection and recognition and multi-user conflict processing are optimized, accurate mapping from gestures to virtual operation is achieved, interaction accuracy and cooperation fluency are improved, and immersive experience of a virtual classroom is enhanced.

The invention provides a gesture recognition method based on an optical fiber sensing technology and deep learning, and relates to the technical field of optical fiber sensing. According to the gesture recognition method, gesture information is captured through speckle image changes of the multimode optical fiber fixed on a glove under different gestures by utilizing the deformation characteristic of the multimode optical fiber, and high-precision gesture classification is carried out in combination with a convolutional neural network (CNN) and a self-attention mechanism. The specific implementation of the method comprises the steps of layout and fixation of multimode optical fibers, speckle image acquisition, data preprocessing, neural network model design and training and the like. Wherein a 635nm semiconductor laser is used as a light source, a CCD camera is used for collecting speckle images in real time, a Python script is used for carrying out optimization processing and data enhancement on the images, a final deep learning model is composed of a feature extraction layer, a global feature modeling layer and a classification layer, and high-precision recognition of gestures is achieved.

The embodiment of the invention discloses a gesture recognition method and electronic equipment. The method comprises the following steps: performing image acquisition on a user gesture through an image acquisition device to obtain a continuous video sequence; performing gesture region extraction on each image frame in the continuous video sequence to obtain a gesture image, and performing feature extraction on the gesture image to obtain spatial stream features; performing optical flow calculation on two continuous frames of images to obtain optical flow images, and performing feature extraction on the optical flow images to obtain time flow features; and fusing the spatial flow features and the time flow features to obtain fused features, and performing gesture classification on the fused features based on a classifier to determine a gesture recognition result. According to the scheme, the space features and the time features can be subjected to cross-modal correlation fusion, so that the two features are complementary in gesture recognition, the motion features can know that the apparent features pay attention to a dynamic area, the border features can correct misjudgment of the operation features, and the accuracy of gesture recognition is improved.

The present invention discloses a method and device applicable to underwater pose recognition, relating to the technical field of underwater detection, including: S1, input an image, and extract visual features through a feature extraction network; S2, construct a position embedding, add by position, and send the result to the transformer encoder E; S3, adopt a pre-trained model CLIP to extract image features; S4, after decoding the output result of E, combine it with the features extracted by CLIP as the true features and input them into the structure of the GAN. The method and device applicable to underwater pose recognition combine the strong representation learning ability of the Transform structure as a visual feature extractor, and then use the generative adversarial network to synthesize visual features of unseen gestures, enabling us to train a gesture classifier using data from both visible and unseen data, overcoming the weaknesses of supervised learning.

In an embodiment, a method includes: obtaining one or more positional time spectrograms of a radar measurement of a scene comprising an object; and based on the one or more positional time spectrograms and based on a feature embedding of a variational auto-encoder neural network, predicting a gesture class of a gesture performed by the object.

The invention relates to the technical field of electromyographic signal processing, in particular to an electromyographic acquisition signal processing method and system for gesture classification. The method comprises the following steps: forming a signal label and an electromyographic signal sample cluster based on a gesture type, electromyographic signal acquisition duration and gesture recognition accuracy; calibrating a cross-cluster error value of the electromyographic signal features; mapping the cross-cluster error value to a two-dimensional coordinate system and adding a point feature value; and generating a to-be-processed sample set through the dual-scale data screening window, judging a gesture type membership degree, and executing visual display. According to the method, the classification precision problem caused by user action habit differences is solved through intelligent data analysis, the accuracy and adaptability of gesture classification are improved, and the method is suitable for scenes such as human-computer interaction and rehabilitation assistance.

The present invention proposes a two-stream cyclic attention dynamic gesture recognition method based on motion guidance. The method includes: using a video codec tool to extract motion vectors, and combining a prediction frame residual analysis mechanism to quantify the dynamic intensity of the hand and generate a motion mask; using a gated network and a graph neural network for the RGB features and the hand motion contour mask to perform gated fusion and graph convolution calculations in sequence; using a dynamic spatio-temporal decay module and a multi-scale wavelet spectrum-spatial block to perform dilated convolution processing, as well as multi-scale frequency domain reconstruction and channel interaction respectively to obtain a gesture classification result. The present invention proposes a cyclic attention mechanism based on a fully convolutional architecture, combined with a learnable time decay strategy, to achieve efficient and low-complexity temporal modeling. At the same time, a wavelet decomposition and reconstruction and multi-scale pooling method are adopted to capture fine-grained hand motion changes, enabling the network to effectively model short-term temporal dependence features and further improving the recognition accuracy of dynamic gestures.

The application belongs to the field of millimeter wave radar perception and human posture recognition, and discloses a non-contact personnel identity and posture recognition method based on motion instability compensation, comprising the following steps: step 1, static background points are adaptively screened, the radar self-motion speed is estimated, and the point cloud coordinates and the micro-Doppler spectrum are compensated; step 2, a GesAuthNet double-branch deep neural network is built to obtain shape parameters and posture parameters; step 3, an identity authenticator is constructed in combination with the shape parameters to authenticate the legitimacy of the user, and a gesture classifier is constructed in combination with the posture parameters to identify the gesture of the user. The application has the advantages of motion instability compensation capability, robust gesture feature extraction capability, high-precision identity authentication and gesture recognition, and strong stability and robustness.