8 results about "Gesture segmentation" patented technology

The invention discloses an infrared gesture detection and segmentation method based on a balance score fusion mechanism, and belongs to the field of gesture recognition, a model adopts the design of a double-flow network structure, and comprises a global motion network, a gesture posture network and a feature fusion module, the global motion network is composed of a plurality of two-dimensional convolutional neural networks, a recognition module, an association module, a one-dimensional convolutional neural network and a bidirectional long-short-term memory network in sequence so as to extract the overall spatial-temporal characteristics of gesture motion; the gesture attitude network is composed of a gesture attitude estimation network based on temperature sensing, a gesture attitude evolution body and an attitude feature extraction network in sequence so as to extract attitude change features of gestures, and the feature fusion module fuses the overall spatial-temporal features and the attitude change features by adopting a balance score; the model provided by the invention can realize accurate gesture boundary detection and segmentation functions, efficiently learn gesture transition features, and meet the gesture segmentation requirements of gesture recognition task automation.

The invention discloses a gesture understanding and robot action generation method and system based on conditional diffusion. The method comprises the following steps: performing gesture segmentation on a panoramic image to obtain gesture sub-images, and reserving the panoramic image as a panoramic scene image; obtaining a gesture visual feature based on the gesture sub-graph; obtaining gesture embedding features based on the gesture sub-graphs; obtaining environment context features based on the panoramic scene graph; fusing the gesture visual features and the environment context features to obtain a fused feature vector; carrying out multi-mode splicing to obtain condition features; and generating an action vector for the mechanical arm of the robot to execute through multi-step iterative denoising by taking the random noise as the beginning through the conditional diffusion model. The robot motion is generated through multi-modal feature extraction, double-branch attention fusion and conditional diffusion generation network and deep combination of gesture vision, three-dimensional key points and scene semantics, and efficient and robust execution from gesture input to robot closed-loop control can be achieved.

This invention discloses a method and system for gesture understanding and robot motion generation based on conditional diffusion. The method includes: segmenting a panoramic image into gesture sub-images and preserving the panoramic image as a panoramic scene image; obtaining gesture visual features based on the gesture sub-images; obtaining gesture embedding features based on the gesture sub-images; obtaining environmental context features based on the panoramic scene image; fusing the gesture visual features and environmental context features to obtain a fused feature vector; performing multimodal stitching to obtain conditional features; and generating motion vectors for the robot's robotic arm to execute using a conditional diffusion model with random noise as the initial source and through multi-step iterative denoising. This invention, through multimodal feature extraction, bi-branch attention fusion, and a conditional diffusion generation network, deeply integrates gesture vision, 3D keypoints, and scene semantics to generate robot motions, enabling efficient and robust execution of gesture input into the robot's closed-loop control.

A method, system, device and medium for gesture segmentation based on graph convolution and attention mechanism, the method comprising: using an RGB image containing a hand as network input, preprocessing an image data set, and dividing the preprocessed image data set into a training set and a test set; segmenting the RGB image using superpixels to obtain an initial segmentation region mask of the RGB image, inputting the initial segmentation region mask of the RGB image into a graph convolution layer, constructing a graph convolution pre-trained network in combination with an attention mechanism model, training the graph convolution pre-trained network, and optimizing network parameters in combination with a cross-entropy loss function to obtain a classification model; the system, device and medium are used to implement a method for gesture segmentation based on graph convolution and attention mechanism; the present invention improves the accuracy of hand categories in images, makes gesture segmentation more accurate, and makes edge segmentation clearer, and reduces the size of model parameter files, making it easier to deploy on hardware devices and having higher operating efficiency.

The application provides a sign language recognition method, aiming at providing a sign language recognition solution for the hearing impaired population. In view of the image background interference, gesture deformation and similar features between classes, the 16-bit depth information is converted into 8-bit depth information, the depth pixel is pseudo-colorized, the sign language gesture is segmented from the background through the color gamut segmentation method, the color fusion is carried out on the gestures with large span through the pixel fusion method, the affine transformation is carried out on the depth image and the color image, the pixel alignment is realized, finally the double-channel feature fusion attention network (DFANet) is trained, the feature fusion is carried out in the later stage of the network, and the classification is carried out through the softmax classifier. Compared with the prior art, the network feature extraction effect of the application is better, the sign language gesture segmentation precision is higher, the network model parameter amount is smaller, the training cost is low, and the operation efficiency is high.

This invention provides a gesture segmentation network device and method based on a multi-branch cascaded Transformer. The device includes a degree convolutional neural network (DCNN) to extract features from the original gesture image to obtain an intermediate feature map. The multi-branch cascaded Transformer module (MBCT) includes multiple cascaded Transformer branches, each of which includes a Patch Partition layer, a Linear Embedding layer, and a Multi-Window Self-Attention Block (MWSA) connected in series. A decoder is used to restore the gesture image to the same size as the original. The segmentation results of this invention have smoother hand edges, stronger ability to remove complex backgrounds, and greater robustness and effectiveness. This invention exhibits high accuracy and strong robustness even under conditions of uneven lighting, complex background noise, and varied gesture shapes.

This application discloses a method and system for gesture feature acquisition and recognition in manned aircraft cockpits, including the following steps: gesture acquisition and processing: adaptively controlling the supplementary lighting component and lens aperture according to the lighting environment to acquire a clear original image of the pilot's hand; enhancing, denoising, and performing analog-to-digital conversion on the original image to generate a digital image signal; data preprocessing: preprocessing the digital image signal to remove noise; gesture segmentation: based on color and depth information, first extracting the gesture region from the depth image, then performing gesture color segmentation in the color space, and then extracting the gesture contour; feature fusion: extracting gesture features including fingertip position, number of fingertips, fingertip angle, and palm center from the gesture contour; recognition result classification: inputting the gesture features into a deep learning-based classification model to obtain the gesture category. This application ensures recognition accuracy in aviation environments.

The invention discloses a lightweight rapid gesture recognition method and system based on a skeleton. The method comprises the following steps: acquiring a to-be-recognized video stream in real time through image acquisition equipment; performing gesture segmentation processing on the to-be-recognized video stream by adopting a gesture segmentation method based on skin color to obtain a gesture segmentation image; performing skeleton extraction processing on the gesture segmentation image by adopting a maximum disc method to obtain a skeleton extraction image; feature extraction is carried out on the skeleton extraction image, skeleton features are obtained, and the skeleton features comprise joint point distance features, joint point direction features and motion trail features; and inputting the skeleton features into a convolutional neural network to realize gesture recognition. According to the embodiment, the problems that a traditional visual recognition model is poor in environmental sensitivity, and a recognition model with high precision faces high computing resource overhead are solved.