74 results about "Hand region" patented technology

The invention relates to the technical field of computer vision and human-computer interaction, and discloses a self-adaptive hand image recognition method and system based on multi-source feature fusion. The method comprises the following steps: acquiring a real-time video stream image from a camera, carrying out various filtering processing on the real-time video stream image, and respectively generating an illumination enhanced image, a background subtraction mask, a skin color area mask and a dynamic motion mask; dynamically adjusting the fusion weight of each filtering processing result according to the scene features of the real-time video stream image, and generating comprehensive filtering output through weighted fusion; performing multi-stage hand segmentation processing on the comprehensive filtering output to obtain an initial hand region mask, performing time sequence smoothing processing, and extracting a maximum connected domain as a final hand region mask; and inputting the final hand region mask into a convolutional neural network for gesture recognition, outputting a prediction probability of each type of gestures, and displaying a gesture classification result in real time. According to the method, the adaptability of hand detection to strong interference backgrounds and variable environments is enhanced.

The invention discloses an XR gesture tracking and recognition method based on multi-view depth fusion. The method comprises the following steps: constructing a virtual camera by adopting a perspective cutting method, and dynamically extracting a hand region after geometric correction through gesture detection and perspective cutting of each frame of image; in combination with internal and external parameters of a camera, through position coding and a multi-head attention mechanism, a geometrical relationship between visual angles is modeled, semantic relevance of multi-visual-angle 3D features is deeply excavated, globally consistent hand representation is constructed, then six-degree-of-freedom information of wrist joints and rotation angles of the joints are output based on a regression device, and forward kinematics and a linear skin algorithm are combined, so that the degree of freedom of the wrist joints is calculated. Generating a complete hand posture and skeleton key point coordinates; and based on the key point position and the rotation angle of each joint, gesture recognition is realized through template matching. According to the method, multi-view image information and hand kinematics constraints are combined, high-precision and high-robustness three-dimensional gesture attitude estimation and classification are achieved while real-time performance is guaranteed, and the method is suitable for various XR scenes.

The application relates to the technical field of image three-dimensional reconstruction. The application discloses a monocular hand-object interaction three-dimensional reconstruction method and device, a terminal and a storage medium, which can improve the quality of a hand-object interaction reconstruction result. The method comprises the following steps: acquiring an initial object grid, an initial hand grid and a target hand-object interaction grid corresponding to a target image; performing hand region and object region extraction processing on the target hand-object interaction grid to obtain a target hand rough region and a target object rough region; performing coarse registration of the initial object grid to the target object rough region by using an object registration algorithm to obtain an optimized object grid; performing coarse registration of the initial hand grid to the target hand rough region by using a hand registration algorithm to obtain an optimized hand grid; and performing parameter adjustment processing on the optimized object grid and the optimized hand grid by using a joint optimization algorithm to obtain a hand-object interaction reconstruction result corresponding to the target image.

The invention provides a dynamic gesture preprocessing and authentication method, a medium and equipment. The method comprises a preprocessing stage and an authentication stage. The preprocessing stage comprises the following steps: carrying out hand region segmentation and skeleton information extraction on input video data; performing standardization processing on the input video data to obtain a standardized image sequence and a corresponding skeleton sequence; the authentication stage comprises the steps of inputting a standardized image sequence and a corresponding skeleton sequence into an appearance and motion network for feature extraction; the appearance and motion network comprises appearance branches and motion branches; the appearance branch extracts appearance features from the standardized image sequence, and the motion branch extracts motion features from the skeleton sequence; and carrying out feature level fusion through a self-adaptive recoupling and fusion mechanism to obtain a final identity feature. According to the method, a double-flow authentication network is adopted, decoupling of appearance and motion features is achieved, targeted extraction and complementary expression of the features are achieved, and therefore the authentication efficiency and accuracy are improved.

The invention discloses a finger joint point spatial position estimation method, a finger joint point spatial position estimation device and head-mounted display equipment. The method comprises the following steps: performing hand recognition on an image acquired by a depth camera through a palm region detection model, and if the image has a hand region, marking the hand region from the image to obtain a hand region image; correcting the hand region image to obtain a distortionless hand region image; identifying the pixel coordinates of the articulation points in the distortionless hand region image and the relative distance of each articulation point through an articulation point detection model; and according to the pixel coordinates of the joint points and the relative distance, calculating through a preset objective function to obtain the absolute distance of the joint points. According to the invention, the calculation amount in the 3D hand joint point estimation process can be reduced, and the technical effects of meeting the high real-time requirement in AR / VR application are achieved.

The invention discloses a gesture recognition method and device based on a pulse camera, a medium and a product, relates to the technical field of gesture recognition, and discloses the gesture recognition method based on the pulse camera, and the method comprises the steps: taking pulse image data collected by the pulse camera as the input of a hand detection network, and obtaining dimension rising feature data based on an expansion convolution layer; expanding the dimension raising feature data in an inverse residual block of the hand detection network based on a target expansion coefficient to obtain an expanded deep convolutional feature, the target expansion coefficient being smaller than a default expansion coefficient of the hand detection network; obtaining a weighted feature map obtained after the expanded depth convolution feature passes through an embedded SE architecture; performing multi-dimensional collaborative hand region directional extraction processing based on the weighted feature map to obtain a hand region of the pulse image data; and identifying the gesture type of the hand region. The stability and accuracy of gesture recognition under a low-computing-power platform are improved.

Systems, methods, devices and non-transitory, computer-readable storage mediums are disclosed for the tracking of interacting hands using a sensor of a wearable multimedia device. In an embodiment, a method comprises: obtaining, with a sensor of a wearable multimedia device, a first frame of two-dimensional (2D) image data and a second frame of three-dimensional (3D) depth data; determining multiple hand regions in the first frame of 2D image data; determining a location of each hand region in the first frame of 2D image data; detecting at least one landmark in each detected hand region; generating a confidence score for each landmark detected in each detected hand region; determining 3D world coordinates for each landmark that is visible in the first frame of the 2D image data; tracking each landmark in 3D world coordinates; and determining an interaction with the hands based on the tracking of each landmark in 3D world coordinates.

The invention discloses a hand detection method and device, a nonvolatile storage medium and electronic equipment. The method comprises the following steps: processing an image by using a computer vision model to obtain a plurality of prediction frames and a left-hand confidence score and a right-hand confidence score corresponding to each prediction frame; marking a left-hand prediction frame and a right-hand prediction frame in the plurality of prediction frames; copying the target prediction box to obtain a first copy and a second copy, marking the first copy as a left-hand prediction box, and marking the second copy as a right-hand prediction box; performing feature modulation processing on the left hand prediction frame set to obtain key point coordinates corresponding to a hand region in a left hand prediction frame; and performing feature modulation processing on the right-hand prediction frame set to obtain key point coordinates corresponding to the hand region in the right-hand prediction frame. The technical problems of detection confusion and key point identity confusion caused by difficulty in accurately distinguishing the left hand and the right hand in hand detection and hand key point positioning are solved.

The invention discloses a hand movement track recognition method, system and device and a medium. The hand motion trail recognition method comprises the following steps: acquiring hand shot image data; the hand shot image data image is input into an improved YOLOv5 gesture detection model to obtain a hand area, the improved YOLOv5 gesture detection model comprises a backbone network, a neck network and a head network, C3 modules in the backbone network and the neck network are replaced with C3Rep modules, and a SimAM attention mechanism is introduced into the backbone network; and according to the hand region, obtaining a hand motion track. By adopting the embodiment of the invention, the hand region can be quickly positioned, and the hand motion track can be accurately identified and fitted, so that the track identification speed and precision can be improved, and reliable data support can be provided for related technologies.

This invention discloses a machine vision-based method and system for monitoring medication dispensing. The method involves acquiring a multi-view polarized image sequence and corresponding depth image sequence of the medication dispensing operation area; performing physical-level specular de-illumination on the multi-view polarized images and identifying transparent material regions to generate multimodal fusion data with transparency priors; detecting the operator's hand posture in real time to generate a dynamic occlusion mask; constructing and training a dynamic neural radiation field, incorporating a temporal de-occlusion prior loss during training to utilize unoccluded spatiotemporal information to recover the visual features of the area marked by the dynamic occlusion mask; and based on the trained dynamic neural radiation field, removing the hand region and rendering a de-occluded and specular-de-illumination virtual monitoring view. This invention solves the problems of target loss, inaccurate measurement, and poor interpretability caused by specular reflection, transparent objects, and dynamic occlusion.

The application discloses a gesture recognition method and a display device. The method uses hand posture confidence to determine whether to simplify the step of determining hand region position information. When the hand posture confidence corresponding to the previous frame of original color image is not less than the preset confidence, the step of determining the second hand information can be simplified, and the time for recognizing the gesture is shortened. The method comprises the following steps: when the obtained original color image is not the first frame of original color image, determining the hand posture confidence corresponding to the previous frame of original color image; if the hand posture confidence is not less than the preset confidence, predicting the hand region position information of the obtained original color image according to the first hand information obtained from the previous frame of original color image; removing the interference pixel points by using the hand region position information to obtain the first depth image; determining the second hand information by using the first depth image; and determining the hand posture information according to the first hand information and the second hand information.

The application discloses a method for determining the centroid of a human hand region and a display device. When a user performs 3D gesture interaction, an image collector collects an RGB image and a depth image. Based on the coordinates of the center point of the human hand detection frame in the RGB image, the coordinates of a target point corresponding to the center point of the human hand detection frame in the depth image are calculated. Based on the coordinates of the target point and the depth value of the human hand region where the target point is located, the coordinates of the centroid of the human hand region in the depth image are calculated. The method and the display device are based on non-aligned RGBD images, do not need complex calculation to realize the alignment function of the RGB image and the depth image, but greatly simplify the calculation method of the centroid of the human hand region in the depth image using the epipolar constraint method, reduce the search range of the corresponding feature points from the RGB image to the depth image, can timely determine the centroid of the human hand region, realize real-time interaction of the 3D gesture algorithm in an embedded terminal, and improve the immersive interaction experience of the user.

The invention relates to a hand-held vacuum cleaner, comprising - a handle (1) with a hand area (11) designed to position a user's hand, - a sensor (3) arranged in the hand area (11) and designed to sense the presence and absence of a user's hand in the hand area (11), - another sensor (4) designed to sense a movement of the vacuum cleaner, and - a control and / or regulating device (6) which is set up and designed to switch on the vacuum cleaner when the sensor (3) senses the presence of the user's hand in the hand area (11) and the further sensor (4) senses a movement of the vacuum cleaner, and to switch off the vacuum cleaner when the sensor senses an absence of the user's hand in the hand area (11) and the further sensor (4) senses a lack of movement of the vacuum cleaner (21).

Embodiments of the present application disclose a gesture recognition method and device, and a storage medium, including: in a case where a to-be-processed image is acquired, processing the to-be-processed image according to a plurality of preset image reduction rules to obtain a plurality of reduced to-be-processed images; performing hand detection on the plurality of reduced to-be-processed images to obtain a hand detection frame; acquiring an image in the hand detection frame from the to-be-processed image to obtain a hand region image; and determining a target gesture in the to-be-processed image according to the hand region image.

The invention provides a double-branch fusion-based dynamic gesture recognition method, a model training method and equipment, and the recognition method comprises the steps: extracting a plurality of target images containing hand targets from target video data, and generating a first image and a second image corresponding to each target image to form a target image sequence, the resolution of the first image is smaller than that of the second image; inputting the target image sequence into a dynamic gesture recognition model to enable the image feature extraction branch, the hand feature extraction branch and the recognition head to obtain gesture category prediction result data corresponding to the target image sequence; and outputting gesture category prediction result data. According to the method and the device, the problems that the model feature extraction capability is reduced and the gesture recognition precision is not high due to the fact that the distance between the hand and the camera is changed to cause the scale inconsistency of the hand region in the existing dynamic gesture recognition mode can be solved.

This invention provides a gesture command recognition method based on an improved contactless data stream network. It utilizes the coordinates of key points in hand data extracted using an improved MediaPipe network algorithm framework based on artificial intelligence. The improvements to the MediaPipe network include increasing the recognition rate of the network model, enhancing the accuracy of key gesture features, and optimizing scene adaptability. The method achieves rapid gesture category determination by "precise hand region cropping and positioning—multi-scale key feature point fusion enhancement—gesture category static and dynamic classification adjustment." This invention addresses the recognition bias caused by excessively small gestures in long-distance photography and detailed gestures in close-up photography by adding a multi-scale feature enhancement module; it also reduces the misjudgment rate of gesture recognition by adding limb nodes and fusing the association between limbs such as the arm and torso; it enhances robustness in complex environments, reducing background interference and occlusion; and it strengthens the motion features between image frames, distinguishing between static and dynamic gestures and improving the dynamic gesture recognition rate.

The embodiment of the application provides a kind of server and portrait segmentation method, hand recognition is carried out to the original picture collected;When there is no hand region, the original picture is input to portrait segmentation network, and first portrait mask chart is obtained;When there is hand region, if hand region and human region coincide, the original picture is input to portrait segmentation network, and first portrait mask chart is obtained;If they do not coincide, hand picture is copied and input to hand segmentation network, and hand mask chart is obtained, and the original picture is input to portrait segmentation network, and second portrait mask chart is obtained;Hand mask chart and second portrait mask chart are fused, and third portrait mask chart is obtained;According to first or third portrait mask chart, portrait part is segmented out.The application trains portrait segmentation network using multi-scale edge supervision, improves image edge segmentation effect, separately processes hand picture through hand segmentation network, improves the integrity of portrait segmentation, and improves the overall experience of user.

The present disclosure relates to an information processing device and an information processing method that make it possible to improve the accuracy of estimating the shape of a person's hand estimated from a captured image of the person who is a subject. In the present invention, a mesh is estimated from an image of the person who is the subject, the mesh is used to set a hand joint parameter to 0 and determine the size from the wrist to the tip of the middle finger when the palm is opened as the size of a hand region, the skeleton of the person is estimated from the image, a center position of the hand region is determined from the joint group corresponding to the skeleton of the person, and the hand shape is estimated on the basis of a hand image obtained by cutting out, with the center position of the hand region as a reference, the size range obtained by multiplying the size of the hand region by a prescribed coefficient. The present disclosure can be applied to virtual viewpoint image reproduction apparatuses.

The application relates to the field of video applications, and provides a video desensitization method and device, electronic equipment and a computer program product. The method comprises the following steps: acquiring video data, identifying a human hand region based on the video data, obtaining N gesture recognition results and M hand contact article recognition results, determining a sensitive region and a target sensitive level according to the N gesture recognition results and the M hand contact article recognition results, and performing desensitization processing on the video data according to the sensitive region and the target sensitive level. The video desensitization method provided in the embodiment of the application can solve the technical problem that a hand contact article or hand operation of a photographed person cannot be desensitized, personal information of the photographed person is easily leaked, and the personal information safety of the photographed person is improved.

The application provides a gesture recognition method, a model construction method, a device and a storage medium. The recognition method comprises: inputting a to-be-recognized image into a pre-constructed gesture recognition model to obtain a first recognition result; in the gesture recognition model, the first recognition model comprises a first feature extraction module, a second feature extraction module, a feature fusion module and a first recognition layer, the first feature extraction module extracts image features of the to-be-recognized image layer by layer to obtain shallow layer features; the feature fusion module determines position features of a user gesture according to the shallow layer features, fuses the shallow layer features and the position features and inputs the fused features into the second feature extraction module; the second feature extraction module is used for feature extraction; and the first recognition layer recognizes image features extracted by the second feature extraction module to obtain the first recognition result. The application can fuse key point features of the to-be-recognized image and the shallow layer features, strengthens hand region features and thus makes the recognition result more accurate.

The application discloses a hand detection method based on a RetinaHand light infrared image, and comprises the following steps: S100, generating a hand region image by using a hand detection network based on RetinaHand; and S200, performing enhancement processing on the generated hand region image. The method has the characteristics of short delay, accurate hand detection positioning and real-time generation support, and can be widely used in natural interaction in the fields of intelligent vehicles, intelligent homes and robots.

This invention relates to the field of wearable device technology, specifically disclosing a wearable hand device, a physiological monitoring component, a signal processing method, and a system. The device includes an elastic wrapper with a palm region and a back of hand region. The palm region has a conductive coupling portion; in the wearing state, both sides of the conductive coupling portion are electrically connected to the user's palm skin and electrodes of an external device, respectively. The back of hand region has a physiological monitoring component, including a physiological signal acquisition module, a signal processing module, and a status indication module. The signal processing module is electrically connected to the physiological signal acquisition module and the status indication module, and controls the status indication module to indicate the physiological state based on the physiological signals acquired by the physiological signal acquisition module. At least one of the elastic modulus, hardness, and thickness of the palm region is lower than the corresponding parameter of the back of hand region. This solution enables physical interaction between the wearable hand device and external devices, as well as real-time physiological state perception under high-load training scenarios.

This application discloses a method, apparatus, and method for training a dynamic gesture detection network. The network training method includes: acquiring multiple gesture videos, each corresponding to a dynamic gesture; performing image processing on each gesture video to determine a first feature map and a second feature map corresponding to each gesture video; wherein the first feature map corresponding to any gesture video is used to represent: motion information between two adjacent frames in any gesture video, or hand contour information in any gesture video; the second feature map corresponding to any gesture video is used to represent: the position information of the hand region in any gesture video; and iteratively training the dynamic gesture detection network based on the first and second feature maps corresponding to each gesture video to obtain a target detection network. This application solves the technical problem in related technologies that training a dynamic gesture detection network requires a large amount of training data and involves a complex training process.

The application relates to a sign language translation method and device, computer equipment and a storage medium. The method comprises the following steps: obtaining a sign language video to be processed; inputting the sign language video to be processed into a hand region detection network to obtain a video stream output by the hand region detection network; inputting the video stream into a sign language recognition module for adaptive recognition processing to obtain a preliminary text representation output by the sign language recognition module; inputting the preliminary text representation into a semantic understanding and text generation module, correcting the grammar of the preliminary text representation by using the semantic understanding and text generation module, and obtaining target language text output by the semantic understanding and text generation module. The application can correct the grammar of the preliminary text representation by using the semantic understanding and text generation module, eliminate the structural difference between sign language and spoken language, make the accuracy of the target language text output by the semantic understanding and text generation module relatively high, and make the expression relatively natural.

This application provides a computer vision-based gesture recognition system, method, device, and medium. For each occluded keypoint, based on a pre-defined spatiotemporal graph model of the target object's hand keypoints, a graph convolutional network is used to extract spatial features of keypoints with high detection confidence in the spatial neighborhood of the occluded keypoint and temporal series features with high detection confidence in historical frames before occlusion. This yields a spatiotemporal collaborative attention map for guiding information completion. Based on the spatiotemporal collaborative attention map, feature propagation and aggregation are performed from the keypoints with high detection confidence to the occluded keypoints to obtain the completed feature representation of the occluded keypoints. The completed coordinates of the occluded keypoints are then output, generating an occlusion-resistant keypoint sequence for the target object's hand region. Based on this occlusion-resistant keypoint sequence, the gesture category of the target object is identified. Using the scheme of this application, robust gesture recognition against occlusion can be achieved.

The invention discloses an exposure adjustment method and device for augmented reality glasses, and relates to the technical field of image processing. The method comprises the steps that an environment image is collected, and whether a hand area exists or not is detected; when the hand region exists, determining a corresponding region of interest, and obtaining a first pixel reference value based on a first gray value of a pixel of the region of interest; when the hand region does not exist, determining a corresponding gray histogram and a weight coefficient corresponding to the second gray value, obtaining a weighted probability density corresponding to the second gray value based on the gray histogram and the weight coefficient, and obtaining a second pixel reference value; and dynamically adjusting exposure parameters according to the current pixel reference value. According to the method, the scene with the hand and the scene without the hand are distinguished, and for the scene with the hand, the local brightness of the hand is focused; for a scene without a hand, the second pixel reference value obtained through weighting the probability density can accurately reflect the global brightness, the accuracy of exposure adjustment is improved, and thus the brightness stability of the image is improved.

The invention discloses a gesture recognition method and system, and relates to the technical field of computer vision. The method comprises the following steps: acquiring an RGB (Red, Green and Blue) image stream of a gesture of a user in real time, and preprocessing the image to obtain a standardized input image; performing character main body detection on the standardized image by using a YOLO target detection model, generating a character bounding box, and performing cutting and magnification processing on the image containing the hand region based on the bounding box; and in the optimized image area, calling a MediaPipe Handds framework to extract two-dimensional coordinates of key skeleton points of the hand, and calculating relative coordinate differences of other skeleton points based on the key points of the palm center to form a standardized gesture feature vector. According to the invention, the problems that the existing gesture recognition technology is often difficult to learn enough feature modes, so that the recognition precision is greatly reduced; meanwhile, for the gesture recognition requirement without peripheral equipment, the problem that in the prior art, when the conditions of different postures, shielding, complex backgrounds and the like are handled, an effective solution is lacked is solved.

The invention is suitable for the technical field of artificial intelligence, and provides a multi-modal depth image three-dimensional hand posture estimation method based on reinforcement learning, and the method comprises the steps: obtaining a depth image; the method comprises the following steps of: constructing a state vector, inputting the state vector into a trained baseline neural network, outputting a heat map and a depth feature representing a joint position, obtaining an initial three-dimensional hand joint coordinate prediction result, constructing the state vector, inputting the state vector into a trained reinforcement learning strategy network, and outputting a refining action of a preset hand joint or a preset hand region; executing a refinement action to update a three-dimensional hand joint coordinate prediction result and related features, and generating a new state vector; repeating the previous steps until a preset termination condition is met; and outputting an optimized three-dimensional hand posture estimation result. According to the method, the estimation precision of difficult hand joints in complex scenes such as shielding and noise can be effectively improved, and meanwhile, the average error of the overall three-dimensional attitude is reduced.

The invention discloses an event-driven robot gesture recognition control method and system. The method comprises the following steps: collecting an asynchronous event flow generated by hand actions of a user through an event camera; dividing a perception picture of the event camera into a plurality of blocks, and determining a player waiting hand region from the asynchronous event stream based on the number of events of each block in the time window; based on the player waiting hand region, extracting position information of hand motion; based on the position information of the continuous multi-frame hand motion, motion angle parameters are obtained through trajectory fitting calculation; and the motion angle parameter is used as a control instruction to be sent to a robot control platform so as to drive a robot joint motor to complete corresponding actions. Compared with the prior art, the method has the advantages that a complete event-driven control link from gesture track fitting to robot joint angle direct mapping is constructed, the problem of high delay in the process of controlling the robot through gesture recognition is solved, and natural and real-time interactive experience is achieved.

The invention relates to a sign language translation method and device, computer equipment and a storage medium. The method comprises the following steps: acquiring a sign language video to be processed; inputting the sign language video to be processed into a hand region detection network to obtain a video stream output by the hand region detection network; inputting the video stream into a sign language recognition module for adaptive recognition processing to obtain a preliminary text representation output by the sign language recognition module; and inputting the preliminary text representation into a semantic comprehension and text generation module, and performing grammar correction on the preliminary text representation by using the semantic comprehension and text generation module to obtain a target language text output by the semantic comprehension and text generation module. The semantic comprehension and text generation module can be used for performing grammar correction on the preliminary text representation to eliminate the structural difference between sign language and spoken language, so that the target language text output by the semantic comprehension and text generation module is relatively high in accuracy and relatively natural in expression.