5 results about "Gesture segmentation" patented technology

The invention discloses an infrared gesture detection and segmentation method based on a balance score fusion mechanism, and belongs to the field of gesture recognition, a model adopts the design of a double-flow network structure, and comprises a global motion network, a gesture posture network and a feature fusion module, the global motion network is composed of a plurality of two-dimensional convolutional neural networks, a recognition module, an association module, a one-dimensional convolutional neural network and a bidirectional long-short-term memory network in sequence so as to extract the overall spatial-temporal characteristics of gesture motion; the gesture attitude network is composed of a gesture attitude estimation network based on temperature sensing, a gesture attitude evolution body and an attitude feature extraction network in sequence so as to extract attitude change features of gestures, and the feature fusion module fuses the overall spatial-temporal features and the attitude change features by adopting a balance score; the model provided by the invention can realize accurate gesture boundary detection and segmentation functions, efficiently learn gesture transition features, and meet the gesture segmentation requirements of gesture recognition task automation.

The invention discloses a gesture understanding and robot action generation method and system based on conditional diffusion. The method comprises the following steps: performing gesture segmentation on a panoramic image to obtain gesture sub-images, and reserving the panoramic image as a panoramic scene image; obtaining a gesture visual feature based on the gesture sub-graph; obtaining gesture embedding features based on the gesture sub-graphs; obtaining environment context features based on the panoramic scene graph; fusing the gesture visual features and the environment context features to obtain a fused feature vector; carrying out multi-mode splicing to obtain condition features; and generating an action vector for the mechanical arm of the robot to execute through multi-step iterative denoising by taking the random noise as the beginning through the conditional diffusion model. The robot motion is generated through multi-modal feature extraction, double-branch attention fusion and conditional diffusion generation network and deep combination of gesture vision, three-dimensional key points and scene semantics, and efficient and robust execution from gesture input to robot closed-loop control can be achieved.

This invention discloses a method and system for gesture understanding and robot motion generation based on conditional diffusion. The method includes: segmenting a panoramic image into gesture sub-images and preserving the panoramic image as a panoramic scene image; obtaining gesture visual features based on the gesture sub-images; obtaining gesture embedding features based on the gesture sub-images; obtaining environmental context features based on the panoramic scene image; fusing the gesture visual features and environmental context features to obtain a fused feature vector; performing multimodal stitching to obtain conditional features; and generating motion vectors for the robot's robotic arm to execute using a conditional diffusion model with random noise as the initial source and through multi-step iterative denoising. This invention, through multimodal feature extraction, bi-branch attention fusion, and a conditional diffusion generation network, deeply integrates gesture vision, 3D keypoints, and scene semantics to generate robot motions, enabling efficient and robust execution of gesture input into the robot's closed-loop control.

This invention provides a gesture segmentation network device and method based on a multi-branch cascaded Transformer. The device includes a degree convolutional neural network (DCNN) to extract features from the original gesture image to obtain an intermediate feature map. The multi-branch cascaded Transformer module (MBCT) includes multiple cascaded Transformer branches, each of which includes a Patch Partition layer, a Linear Embedding layer, and a Multi-Window Self-Attention Block (MWSA) connected in series. A decoder is used to restore the gesture image to the same size as the original. The segmentation results of this invention have smoother hand edges, stronger ability to remove complex backgrounds, and greater robustness and effectiveness. This invention exhibits high accuracy and strong robustness even under conditions of uneven lighting, complex background noise, and varied gesture shapes.

This application discloses a method and system for gesture feature acquisition and recognition in manned aircraft cockpits, including the following steps: gesture acquisition and processing: adaptively controlling the supplementary lighting component and lens aperture according to the lighting environment to acquire a clear original image of the pilot's hand; enhancing, denoising, and performing analog-to-digital conversion on the original image to generate a digital image signal; data preprocessing: preprocessing the digital image signal to remove noise; gesture segmentation: based on color and depth information, first extracting the gesture region from the depth image, then performing gesture color segmentation in the color space, and then extracting the gesture contour; feature fusion: extracting gesture features including fingertip position, number of fingertips, fingertip angle, and palm center from the gesture contour; recognition result classification: inputting the gesture features into a deep learning-based classification model to obtain the gesture category. This application ensures recognition accuracy in aviation environments.