34 results about "Hand skeleton" patented technology

The invention provides a self-supervised continuous 3D hand posture tracking method based on a lightweight inertial measurement unit, and the method specifically comprises the following steps: firstly, obtaining a hand motion signal by using a lightweight consumer-level IMU, and extracting features by using a multi-stage neural network containing a bidirectional long-short term memory (Bi-LSTM) module; the global displacement and orientation of the hand are processed through a wrist posture estimation module, and three-dimensional coordinates of hand skeleton points are solved and calculated by means of a finger motion chain; secondly, a self-supervised completion network is realized by combining a random mask and dual loss, and the features are processed to complete complete attitude reconstruction; and finally, combining network output with kinematics prior, generating a three-dimensional grid with reasonable anatomy, solving the problem that joint deformation is inconsistent with torque, and realizing accurate acquisition of a three-dimensional hand posture. For a dynamic hand motion tracking task, the method can adapt to a sparse IMU deployment scene and realize continuous high-precision tracking; the real-time performance demand of daily interaction can be met, and a low-cost solution can be provided for the fields such as medical rehabilitation and virtual interaction which have strict requirements on attitude precision.

The invention discloses a piano teaching interaction method based on a skeleton point image recognition technology. The method comprises the following steps: S1, collecting video data of hand actions when a user plays a piano; s2, carrying out preprocessing on the video data; s3, extracting hand skeleton point data by using the skeleton point extraction model to obtain a skeleton key point data set; s4, training a hand action recognition network model by using the extracted skeleton key point data set and the loss function; and S5, inputting the real-time output of the skeleton point extraction model into the trained hand action recognition network model, and detecting the correctness of the hand posture. According to the method, the motion information of the hands of the piano player can be captured and analyzed in real time with high precision, and targeted and personalized teaching feedback can be realized through deep mining of skeleton point data, so that the interactivity and teaching efficiency of piano teaching are improved.

This utility model discloses a handrail frame with steering adjustment function, including a base structure, a rotation adjustment structure installed on the base structure, a mounting block with a connecting rod fixedly mounted on its surface, and a concave cylindrical block fixedly mounted on the top of the mounting block. A limiting hole is formed on the inner side of the concave cylindrical block, and a guide annular groove is formed along the limiting hole inside the concave cylindrical block. The rotation adjustment structure includes a first handrail frame body with a matching rod fixedly mounted at its bottom, and a convex round block fixedly mounted on the surface of the matching rod. A connecting block is fixedly mounted on one side of the first handrail frame body, and another concave cylindrical block is fixedly mounted on the top of the connecting block. The concave cylindrical block is rotatably connected by another matching rod, and the other matching rod is fixedly mounted to the inner side of a second handrail frame body through a U-shaped groove. This handrail frame with steering adjustment function achieves both inner opening and outer fixing through the concave cylindrical block.

The invention relates to the technical field of simulation, in particular to a finger motion trail simulation method and device, equipment, a medium and a program product, and the method comprises the steps: obtaining a hand skeleton image and a hand motion video of a detected target; constructing a three-dimensional hand simulation model according to the hand skeleton image; extracting a plurality of video frames from the hand action video, extracting a two-dimensional coordinate of at least one position point from each of the plurality of video frames, converting the hand action video into a three-dimensional coordinate based on a pre-established coordinate mapping relationship, and forming time sequence data according to the position point of the three-dimensional coordinate; and driving the three-dimensional hand simulation model to execute a simulation action by using the time sequence data, and generating a finger motion track of the tested target according to an execution result of the simulation action. Therefore, the problems of high acquisition cost, poor universality and the like of the finger motion trail in the prior art are solved.

The application provides a hand action interaction rendering method and device, a storage medium and electronic equipment, and applies to the technical field of visual rendering.The target hand model provided by the application is obtained by binding a first hand model and a second hand model on the same hand skeleton hierarchical structure, the geometric size of the second hand model is greater than that of the first hand model, the first hand model is a hand detail model used for hand rendering, and the second hand model is a hand simplified model used for physical interaction collision detection.Collision between the hand simplified model and objects in a virtual environment is detected to determine the mode in which the target hand model is located.In a free space mode, a skeleton pose is calculated by using motion capture data and is synchronized to hand detail model rendering.In a physical interaction mode, a more accurate pose is obtained according to vertex displacement of the hand simplified model and is synchronized to hand detail model rendering, so that penetration of the hand model into virtual objects is effectively prevented, and the authenticity and reliability of hand actions are improved.

The application provides an automatic positioning handrail framework welding tool, and relates to the technical field of automobile handrail framework manufacturing. The tool support is provided with an inner table groove in the upper portion, and a transposition servo motor is arranged in the inner table groove. The transposition servo motor is connected with and drives a rotary disc type welding table. Six handrail framework positioning tables are movably arranged on the rotary disc type welding table. A framework rod limiting seat is fixedly connected to the inner end of the upper surface of the handrail framework positioning table. The support rod and the rotary base shell are fixed and properly arranged by the rod body side clamping plate and the round head side clamping plate, respectively. The automatic butt joint, proper arrangement and fixation of the support rod and the rotary base shell are achieved. Manual operation is not needed, time and labor are saved, the problems of low placing efficiency and high deviation rate of manual placing of the handrail framework are solved. After the placing of the handrail framework is completed, the clamping mechanism in the tool is controlled to clamp the handrail framework. The finished product handrail framework after welding and assembly needs to be manually released from clamping and discharged, which is time-consuming and labor-consuming.

The application provides a three-dimensional gesture tracking method based on an RGB camera, which comprises the following steps: taking the RGB camera as an input, performing standardization processing on a picture, and sending the picture into a three-dimensional joint detection module to extract image features, predict a preliminary three-dimensional joint and calculate a hand skeleton length; using a particle swarm optimization algorithm to iteratively fit the hand skeleton length and a MAMO hand model, finding an optimal hand shape and obtaining a new three-dimensional joint; sending the new three-dimensional joint into an inverse kinematics module to solve posture parameters required by the MAMO model; using a MANO model to combine the optimal hand shape and the posture parameters θ to obtain final joints and grid vertices; and performing reconstruction rendering on the obtained grid vertices and joints to output a final three-dimensional hand real-time tracking posture. The method provided by the application has high prediction accuracy and high three-dimensional gesture tracking effect precision, and the tracking process and result are easy to realize.

The invention discloses a mixed reality physical entity interaction method and system based on hand skeleton tracking, and belongs to the technical field of human-computer interaction. The problems that in existing mixed reality interaction, in-air gestures are low in precision and prone to fatigue, and a special controller is not portable and high in cost are solved. The method comprises the following steps: acquiring hand three-dimensional skeleton node data when a user grabs a passive physical entity through head-mounted computing equipment; at least two key skeleton nodes are selected, and control parameters representing the real-time relative space relation of the key skeleton nodes are calculated; and establishing mapping between the parameter and the target attribute of the virtual interaction element so as to drive virtual interaction in real time. According to the method, any common physical entity is temporarily converted into a high-precision interaction medium, and the method is mainly used for realizing natural and low-cost fine control with physical feedback in mixed reality application.

The invention relates to the technical field of motion capture technology, and discloses a human motion capture system based on multi-modal synchronous acquisition, comprising: a motion capture sampling terminal for collecting human motion data including a human motion scene; the optical inertial tactile glove device is used for collecting human hand action data and fingertip tactile data; the time synchronization module generates and outputs a unified synchronous trigger signal; the data processing work station receives, synchronizes and fuses data of the sampling terminal and the optical inertial touch glove device to generate unified whole body and hand skeleton information; the compensation adjustment module is in communication connection with the data processing workstation, receives specified parameters fed back by the data processing workstation, and adjusts the ambient light intensity and the triggering frequency and the triggering time of the synchronous triggering signal based on a preset comparison strategy; the specified parameters comprise an ambient light intensity coefficient, a synchronization rate of whole body and hand skeleton information and actual human body actions, and a movement speed and a movement acceleration of a specified position of a human body.

The invention discloses a hand-containing picture generation method and device, electronic equipment and a storage medium, and the method comprises the steps: generating a first picture based on text information, carrying out hand key point detection processing on the first picture to obtain first skeleton information of a first hand, the first skeleton information is matched with second skeleton information of hand skeletons in a preset skeleton library, a target hand skeleton is determined from the hand skeletons in the preset skeleton library, the target hand skeleton serves as auxiliary guiding information, a second picture is generated based on the text information, and the second picture comprises a second hand. The similarity between the second hand and the first hand meets the preset similarity, and the abnormal rate of the first hand is larger than that of the second hand. According to the embodiment of the invention, the picture is regenerated by using the matched skeleton, so that the possibility of hand malformation is reduced, the consistency of the picture is ensured, and the color and edge problems caused by local generation cannot be caused.

The invention provides a hand action interaction rendering method and device, a storage medium and electronic equipment, and is applied to the technical field of visual rendering. A target hand model provided by the invention is obtained by binding a first hand model and a second hand model on the same hand skeleton hierarchical structure, the geometric size of the second hand model is greater than that of the first hand model, and the first hand model is a hand detail model for hand rendering. The second hand model is a hand simplified model used for physical interaction collision detection. And detecting collision with an object in the virtual environment through the simplified hand model, and judging the mode of the target hand model. In the free space mode, skeleton postures are calculated through motion capture data and synchronized to hand detail model rendering, and in the physical interaction mode, according to vertex displacement of a hand simplified model, more accurate postures are obtained and synchronized to hand detail model rendering, so that the hand model is effectively prevented from penetrating through a virtual object, and the user experience is improved. And the authenticity and reliability of hand actions are improved.

The application discloses a kind of intelligent animation modeling methods based on three-dimensional technology, it is related to computer graphics and three-dimensional animation production technical field, the application reconstructs hand-object interaction logic by three-dimensional technology, realize natural fine motion generation;In virtual anchor end, such as coffee cup scene, the spatial relationship of hand skeleton and prop point cloud is quantified as the topological connection diagram that can be calculated, rigid connection edge ensures the stable grip of functional area, fingertip and cup body have no suspended gap;Elastic adsorption edge allows flexible contact of non-functional area, when palm pressure changes, deformation gradient field drives prop surface microdeformation, synchronous joint torque dynamic adjustment, avoid hand rigid distortion when coffee pours in traditional scheme. The collaborative design of graph convolution network and physical engine further eliminates the risk of wearing mold: equidistance constraint of prop rigid area retains original topological structure, and soft contact area absorbs collision energy through deformation gradient field.

The present application relates to a kind of handrail framework assembly welding fixture, including the A clamping unit for the clamping of slide rail support sheet metal part, the B clamping unit for the clamping of left handrail vertical support piece and right handrail vertical support piece, the C clamping unit for the clamping of handrail horizontal support square steel, handrail vertical support square steel, outer handrail connecting plate, left handrail vertical fixed square steel, the D clamping unit for the clamping of lower end framework connecting plate, the E clamping unit for the clamping of right handrail vertical fixed square steel, the F clamping unit for the clamping of outer plate fixed support steel wire, the G clamping unit for the clamping of lower end framework connecting steel wire, the H clamping unit for the clamping of handrail support steel wire.The above scheme provided by the present application can effectively clamp and position each component of handrail framework assembly reliably, so as to reliably weld and connect each component on the welding fixture, improve welding efficiency and reduce labor intensity.

The application discloses a kind of based on semi-supervised 3D skeleton estimation method, device, equipment and storage medium, it includes: obtaining to be processed image, the human skeleton or hand skeleton is included in the to-be-processed image;The to-be-processed image is input into skeleton estimation model, obtains the target three-dimensional skeleton coordinates corresponding to the human skeleton or the hand skeleton, wherein the skeleton estimation model includes by extracting features to training image data, the skeleton point uvd coordinate, the skeleton point space coordinate and the hidden variable camera mapping image parameter obtained are trained neural network by preset loss function, the skeleton point uvd coordinate includes the image coordinate of skeleton point and the depth value corresponding to the image coordinate of skeleton point.It can meet the use demand of user, while, greatly save the cost of 3D skeleton data acquisition and annotation.

The invention is suitable for the technical field of terminals, and provides an air gesture recognition method, electronic equipment and a computer program product. And inputting the hand shape categories, the hand skeleton point information and the joint distance vectors corresponding to the M frames of target images including the continuous N frames of target images into a dynamic gesture recognition model for processing to obtain confidence vectors of the air gestures, and determining the categories of the air gestures according to the confidence vectors. The dynamic gesture recognition model is obtained by training a large amount of different sample data, so that the air gesture recognition method can have relatively high generalization performance when the dynamic gesture recognition model is adopted to recognize the air gesture category; the method can cope with various air-crossing gestures with different hand shape change characteristics made by different users for the same air-crossing gesture, and the use experience of the air-crossing gesture function is improved.

The application relates to the technical field of motion capture technology and discloses a human motion capture system based on multi-modal synchronous acquisition, which comprises a motion capture sampling terminal, which collects human motion data containing a human motion scene; an optical-inertial-tactile glove device, which collects human hand motion data and fingertip tactile data; a time synchronization module, which generates and outputs a unified synchronous trigger signal; a data processing workstation, which receives, synchronizes and fuses data from the motion capture sampling terminal and the optical-inertial-tactile glove device, and generates unified whole-body and hand skeleton information; and a compensation adjustment module, which is in communication connection with the data processing workstation, receives specified parameters fed back by the data processing workstation, adjusts the environmental light intensity and the trigger frequency and trigger time of the synchronous trigger signal based on a preset comparison strategy; the specified parameters include an environmental light intensity coefficient, a synchronization rate of the whole-body and hand skeleton information and actual human motion, and motion speed and motion acceleration of a specified position of the human body.

The invention provides a method, a device and equipment for automatically binding a target 3D object and a digital human hand. The method comprises the following steps: determining a hand skeleton of a preset digital human and a target 3D object to be bound with the hand skeleton; based on the geometric features of the target 3D object, determining a target gesture required by the hand skeleton to hold the target 3D object; determining the current pose information of the target 3D object, and determining the current reference position of the hand skeleton; based on the current pose information of the target 3D object and the current reference position of the hand skeleton, determining displacement information of the target 3D object moving to the reference position; according to the target gesture, generating a hand closing simulation track of the hand joint; and based on the hand closed simulation trajectory, performing collision detection on each hand joint in the hand skeleton to obtain a relative position relationship between each hand joint in the hand skeleton and the target 3D object when the hand skeleton holds the target 3D object with the target gesture.

The application discloses an immersion interactive method and device for appreciating artifacts, and relates to the technical field of human-computer interaction. The method comprises the following steps: in an immersive digital experience cabin, the three-dimensional coordinates of the user's double pupils and the hand skeleton node coordinates are collected, the gesture instruction is analyzed, and the three-dimensional space viewpoint parameter set of the next moment is predicted; in a virtual engine scene, the UV texture coordinates are obtained by ray intersection, the three-dimensional region of interest is demarcated, the candidate tile set and the tile topology feature matrix are retrieved and generated; based on the pre-constructed fitness function, the improved tornado optimization algorithm is used for solving, and the global optimal tile loading sequence is output; the naked-eye light field rendering and the three-dimensional space multi-modal superposition are performed on the global optimal tile loading sequence, and the fusion picture of the artifacts and the suspended interpretation is presented on the naked-eye 3D cabin screen. The application solves the problems of viewpoint prediction lag, texture tile loading leading to video memory overflow or picture damage, and lack of spatial multi-modal fusion in the prior art.

The application discloses a control method and device of a humanoid robot, and relates to the technical field of robots, and comprises the following steps: acquiring three-dimensional position data of hand skeleton key points of a target operator and pose data of a palm root skeleton of the target operator; determining a skeletal direction vector of each finger joint of the target operator according to the three-dimensional position data, and determining each target finger joint angle of a dexterous hand of a humanoid robot to be controlled according to the skeletal direction vector; generating an upper limb joint angle reference track of each upper limb joint of the humanoid robot to be controlled according to the pose data, and generating a joint driving instruction for controlling the motion of the upper limb and the dexterous hand of the humanoid robot to be controlled according to the target finger joint angle and the upper limb joint angle reference track. The application improves the grasping posture reproduction accuracy of the humanoid robot, and enhances the coordination between the grasping action and the arm motion.

The utility model discloses a simulated hand visual sensing teaching aid which comprises a hand model, the hand model comprises a hand skeleton, silica gel covers the outer side of the hand skeleton, a sensor is installed between the hand skeleton and the silica gel, the sensor is connected with a processing device through a data line, a microprocessor is arranged in the processing device, and the microprocessor is connected with the hand skeleton. The side edge of the processing equipment is electrically connected with a display screen; the hand model, the display screen and the processing equipment are distributed on the support frame from top to bottom; according to the utility model, the hand model is arranged, the inner side of the hand model is provided with the sensor, the sensor is electrically connected with the processing equipment, the processing equipment is used for receiving signals transmitted by the pressure sensor and analyzing and processing the signals, and the processing equipment is connected with the display screen for displaying data information, so that accurate positions of acupuncture points of students can be visually prompted; quantized data are provided for hand massage teaching, and the teaching efficiency is improved.

This application discloses a method, apparatus, electronic device, and storage medium for assessing hand contact risk. The method includes: acquiring video data of an indoor scene using a depth camera, wherein the video data includes two-dimensional color images and depth distance data; calculating the coordinates of 25 human skeleton points and up to 21 key hand skeleton points in a first video stream and a second video stream, respectively, as well as the ROI of the object for assessing surface contact risk and the mask coordinates of the object's surface within the ROI, thereby identifying the contact risk value of each person and the contact exposure dose value of any object in the indoor scene at any given time. This solves the problems of related technologies being unable to quantify the hand contact risk of personnel in indoor scenes and quantify the contact exposure dose of object surfaces in specific scenarios.

The utility model belongs to the technical field of armrest framework machining, and particularly relates to drilling equipment for machining an automobile seat armrest. A shifting plate is arranged at the top of the workbench, a positioning assembly is arranged at the top of the shifting plate, a clamping assembly is arranged on the outer side of the shifting plate, and an adjusting assembly is arranged at the bottom of the shifting plate; the positioning assembly comprises a cross-shaped moving block, the cross-shaped moving block is arranged in the shifting plate and is in sliding connection with the shifting plate, a spring is fixedly connected to one side of the cross-shaped moving block, the other end of the spring is fixedly connected with the shifting plate, and a clamping plate is fixedly connected to the top of the cross-shaped moving block; by means of the handrail drilling device, handrail positioning is more convenient and faster, handrails of different sizes can be clamped and fixed, drilling machining of the surfaces of the handrails in multiple directions can be achieved, and the handrails do not need to be taken down for repositioning.

The invention relates to the technical field of man-machine operation interaction, and discloses a man-machine operation interaction method and system based on vision, and the method comprises the steps: collecting an original image sequence containing a hand, carrying out the light correction and background separation processing, and extracting the coordinates of a hand skeleton node to form initial posture data; performing normalization and denoising processing on the hand key points based on the initial attitude data, and constructing static feature data used for describing a hand structure state; further calculating displacement change and motion trend between adjacent frames to obtain dynamic track characteristics reflecting hand motion behaviors; performing time sequence segmentation based on the dynamic trajectory features, and fusing with the static features to determine a comprehensive hand action mode; and performing intention analysis on the action mode, generating an initial semantic tag in combination with an instruction mapping rule, confirming semantic accuracy through track matching and posture verification, and finally outputting a composite gesture control instruction according to a semantic mapping relationship. According to the method, the accuracy and consistency of composite gesture semantic recognition can be improved.

The application belongs to the technical field of human-computer interaction, and discloses a digital twin WEB application air gesture interaction method and system; the method comprises the following steps: collecting a real-time video stream of a camera of a user terminal device, and preprocessing the real-time video stream; obtaining X hand region frames; inputting the X hand region frames into a preset hand key point detection model based on a generative adversarial network to obtain hand key point coordinate data; obtaining a hand skeleton model based on the hand key point coordinate data; obtaining joint angles and hand motion trajectories according to the hand skeleton model; mapping the hand skeleton model to a digital twin virtual scene, presetting n gesture instruction modes in the digital twin virtual scene, and matching the hand skeleton model, the joint angles and the hand motion trajectories with the gesture instruction modes; the method provides personalized and expandable gesture interaction capabilities, and enriches the mode of human-computer interaction.

The invention discloses a general dexterous hand gesture control method based on biological perspective agency, and aims to realize decoupling of a control strategy and a specific hand skeleton by introducing a flexion angle and an abduction angle with definite biological semantics as an intermediate agent layer. The same set of control algorithm can adaptively drive dexterous hand models with different forms and different sizes; meanwhile, by utilizing a real-time state fitting technology based on a virtual sensor, a natural dexterous hand grabbing gesture which accords with a biological movement rule and is free of mold penetration can be quickly generated under the condition that long time sequence dependence is not needed, so that gesture control with high universality, high real-time performance and good interpretability is realized.

This invention discloses a digital opera generation and participation system supporting multimodal interaction, comprising an interactive device and a multimodal interaction subsystem. The interactive device includes: a light-shielding shell and a conical light outlet, used to define a conical area along the projection direction as a identifiable interactive area; and a motion-sensing control module, used to collect hand skeleton and motion trajectory information within the identifiable interactive area. The multimodal interaction subsystem includes: a facial recognition module, a digital opera character generation module, an interactive storyline control module, and a rendering engine execution module. This invention has the following advantages: 1. It accurately captures user hand movements, allowing users to select storyline paths, enhancing the interactivity and immersion of digital opera performances, and strengthening the gamified narrative capabilities of opera; 2. The interaction method is simple and intuitive, reducing the system threshold and difficulty of use, and has stronger scalability and universality.

The invention discloses a dynamic three-dimensional image fragmentation display system, which comprises an image input and segmentation module for receiving a source two-dimensional image and segmenting the image into a plurality of image fragments; the three-dimensional space mapping and attribute enabling module is used for distributing an initial position in a three-dimensional coordinate system for each segmented image fragment and endowing each segmented image fragment with an independent rotation angle, an independent scaling and an independent transparency attribute; the dynamic effect generating and rendering module is used for simulating dynamic behaviors of the image fragments by utilizing a physical engine or a particle system; the gesture recognition and interaction processing module is used for analyzing the hand skeleton data in real time and recognizing a predefined gesture instruction; and the interaction instruction execution module is used for carrying out real-time intervention on the dynamic simulation process according to the analyzed gesture instruction. According to the method, the two-dimensional image can be deconstructed into the fragments capable of dynamically suspending in the three-dimensional space, and the immersive interactive experience is realized in combination with the gesture recognition technology.

The invention relates to the technical field of virtual reality gesture recognition, and provides a Meta XR two-hand cross gesture recognition method based on a virtual point ray, and the method comprises the steps: operating a target application in a virtual reality environment of Meta XR equipment; calling a gesture recognition interface provided by the virtual reality environment, collecting hand skeleton information data of a user, and extracting skeleton nodes of left and right palm centers; taking palm center skeleton nodes of the left hand and the right hand as virtual points, taking forward directions of local coordinate systems of the virtual points as directions of palms, adopting a vector point multiplication method to perform vector operation on the forward directions of the left hand and the right hand, and when a point multiplication result is smaller than a preset threshold value, judging that a double-hand cross gesture exists, and triggering preset interaction feedback in the virtual reality environment. According to the method, the virtual point ray is generated based on the direction information of the palm center skeleton node without depending on the preset gesture configuration of Meta XR, and the two-hand cross gesture is judged through the vector point multiplication operation, so that the accuracy of two-hand interaction recognition is effectively improved.

The application relates to the technical field of fingertip detection, and discloses a real-time fingertip detection method and equipment based on a lightweight model, which comprises the following steps: collecting hand image data and constructing a hand skeleton constraint relationship database; performing hierarchical feature extraction and feature hierarchical adaptive fusion through a mixed density encoder network to generate a comprehensive feature map; performing feature mapping fusion to output initial fingertip position coordinate data; adopting a model lightweight technology to optimize the network structure of the mixed density encoder network to generate a lightweight fingertip detection model; performing interframe difference detection and adaptive confidence threshold adjustment to output a real-time updated fingertip accurate position coordinate set, thereby effectively suppressing the jitter phenomenon of single-frame detection results, providing a smooth and continuous fingertip trajectory, and improving the naturalness and comfort of human-computer interaction.