124 results about "Joint coordinates" patented technology

The invention discloses an attitude detection method using a deep convolutional neural network, and is suitable for being performed in computing equipment. The method comprises the steps that a data set is divided into training and testing and preprocessed; identification learning model training of human body joint feature areas is performed so as to identify the learning network of human body joint image areas; joint coordinate positioning learning model training is performed; detection image size preprocessing is performed, and the images requiring human body attitude identification are adjusted into the size of the network input requirements; identification of the image joint areas is performed through the network and corresponding rectangular areas are defined to be saved as sub-images; the acquired sub-images act as the input to be transmitted to the joint coordinate positioning learning model to perform joint coordinate acquisition; and the acquired joint points are connected according to the human skeleton model so as to form human body attitude description. The invention also provides storage equipment and a mobile terminal.

The invention relates to an implementation method for workpiece grasping of an industrial robot based on visual positioning, which comprises the steps that a workpiece image is acquired through a fixed global CCD camera, and workpiece image information is transmitted to a robot control system through an Ethernet interface; the robot control system processes the workpiece image and acquires workpiece position vector information; the robot performs Cartesian and joint coordinate transformation according to the workpiece position vector information so as to realize positioning and grasping of a tail-end claw for a workpiece. The invention provides a workpiece position information calculation method, which is characterized in that contour region screening is performed on all detected images when contour detection is performed on workpiece images, isolated and small-segment continuous edges are deleted, non-target contours are removed, and the target contour identification accuracy is improved; and meanwhile, when workpiece position information is calculated, judgment is performed on a long side of a workpiece, and the robot is controlled to grasp the long side of the workpiece, so that a failure in grasping caused by short side clamping of the robot is avoided, and the grasping efficiency is improved.

The invention provides a method for positioning three-dimensional human body joints in monocular color videos. The method comprises the following steps of: S1, constructing a configurable depth model and importing time sequence information in the depth model; S2, collecting training samples and learning parameters of the depth model by utilizing the training samples; and S3, initializing the depth model by utilizing the parameters learnt in S2, and converting monocular color video data needing three-dimensional human body joint positioning into a plurality of frames of continuous two-dimensional images, inputting the images into the depth model to carry out analysis, and aiming at each frame of two-dimensional image, outputting three-dimensional human body joint coordinates of figures in the image. According to the method, a deep-level convolutional neural network is constructed by utilizing deep learning so as to automatically learn effective spatial-temporal features from a lot of training samples without depending on prior conditions of artificial design and human body joint structural constraints; and through the learnt effective features, the human body joint positions are directly regressed.

The invention discloses a time sequence behavior detection method based on 3D human body key points, and the method comprises the steps: taking video data as input, and converting the video data intocontinuous frame images through data preprocessing;c arrying out Feature extraction by using a multilayer CNN network, and detecting a boundary frame of a personnel target in the image; secondly, through body part positioning and correlation degree analysis, acquiring 2D human body key point coordinates and mapping from 2D human body key points to achieve 3D key points by constructing a key pointregression network; inputting the 3D joint coordinates into a space-time diagram convolutional network to carry out frame-level action recognition and classification on the whole video sequence, and grouping adjacent frames of the same label to obtain action proposal segments with different granularities; andprecisely correcting the time boundary of the action through fine-grained integrity filtering, so that the time sequence behavior detection in a complex scene is realized. According to the method, more valuable information can be analyzed from the 3D data, and the time sequence behavior detection and positioning precision is remarkably improved.

The present invention provides a human body posture detection method and a device. The method comprises the steps of receiving the first depth image of a human body, converting the first depth image into a three-dimensional cloud model, carrying out optimized fitting on a human body posture bubble model obtained in advance and the three-dimensional cloud model, wherein the human body posture bubble model is obtained according to a human body skeleton joint detected in advance, obtaining the human body skeleton joint coordinate in the first depth image according to an optimized fitting result, and obtaining the posture of the human body according to the human body skeleton joint coordinate in the first depth image. By using the scheme provided by the invention, the processing calculation of image recognition and classification of each depth image is not needed, the human body processing efficiency is improved, and the computational resources are greatly saved.

The invention aims to provide a Kinect-based motion sensing-control method for manipulator comprising the following steps of obtaining the three-diemnsional coordinates of five joints of the right upper limb of a human body by a Kinect sensor; smoothing the obtained five joint coordinate data of the right upper limb by a double-exponential filtering algorithm; constructing vectors in the three-dimensional space coordinate system of the Kinect by utilizing the smoothed 5 joint coordinates of right upper limb, and obtaining the angles of the right upper limb by calculating the vector included angles, wherein the angles of the right upper limb comprise a right shoulder joint angle, a right elbow joint angle and a right wrist joint angle; fusing the angle information to form a data package, adding with a data package head and a checksum, and sending to a robot by a wireless serial port to perform manipulator control. The Kinect-based motion sensing-control method for the manipulator can finish the flexible and accurate control on the manipulator and the moving robot by accurate identification on the human body actions, thus enabling the interaction between people and the robot to be more friendly and improving the intelligence of the robot.

The invention discloses a robot motion path off-line programming method and system. Robot kinematics constraint conditions, a robot, a robot tooling three-dimensional model, a robot work object three-dimensional model and a numerical control code file are loaded in a robot three-dimensional virtual environment; Cartesian coordinate system path data of a robot work object are acquired through the numerical control code file, and the Cartesian coordinate system path data of the robot work object are converted into robot joint coordinate path data; robot motion path simulation is performed, the abnormal condition and security of the robot joint coordinate path data are detected, edited and modified, and the robot joint coordinate path data are converted into corresponding robot motion path programs on the condition that no abnormal condition occurs and the security meets the conditions. According to the robot motion path off-line programming method and system, robot complex motion curve path programming can be achieved, and the advantages of being easy to operate, high in working efficiency and the like are achieved.

The invention belongs to the technical field of automatic identification and discloses a gesture estimation method based on a parallel convolution neural network. The method comprises the steps of (S1) using an RGB-D camera to obtain a color image and a depth image, (S2) identifying a position of a hand in the color image according to a hand classification module obtained through training in advance, (S3) finding a corresponding position of the hand in the depth image according to the corresponding position of the hand in the color image, (S4) converting a separated depth image into a gray image, and then inputting the gray image into a parallel convolution neural network gesture estimation model to carry out identification so as to obtain multiple joint coordinates which can express hand gestures, and (S5) outputting a gesture result. The invention provides an end-to-end gesture estimation network architecture, the complexity of an algorithm is reduced, the network convergence speed is greatly improved by a parallel structure, the training time is greatly saved, and improves the efficiency is improved.

The invention discloses a rail transit abnormal person detection method based on action recognition, and belongs to the field of aviation monitoring. The method comprises: inspecting railways by usingan unmanned aerial vehicle, extracting frames of videos, training and using an SSD detection model, obtaining position information of each person in each video frame image, intercepting a local areacontaining the person, and a key point detection model and a Reset-are detected. The 18-clothes classification model is trained; predicting joint coordinates of each person by using the key point detection model, forming a human skeleton sequence by using the joint coordinates in a certain time period, inputting the human skeleton sequence into an LSTM action recognition model, and recognizing theaction category of each person; and performing clothing classification of the personnel through the Resnet-18 clothing classification model; and judging whether the person is a worker or not according to the action type of each person and the corresponding appearance clothes. The method solves the problem of high cost of traditional manual inspection or roadbed inspection, and has high practicability and robustness.

The invention discloses a high-accuracy calibration method of a joint coordinate measuring machine. The method comprises the steps that a to-be-calibrated machine is placed in a measuring space of a reference machine by taking the six-freedom-degree joint coordinate measuring machine as the to-be-calibrated machine and a higher-accuracy three-coordinate measuring machine as the reference machine; calibration plates with taper holes are sequentially placed according to six bisectrices of a circle; the measuring machine measures the taper holes in the calibration plates with a plurality of poses in each position; each pose corresponds to a measuring coordinate of one taper hole under a to-be-calibrated coordinate system and a group of joint angle values; the high-accuracy three-coordinate machine measures coordinates of the taper holes in the positions under a reference coordinate system; the joint angle values and reference coordinates corresponding to measuring points serve as sampling data; a multi-target search problem is converted into a single-target nonlinear programming problem based on the sampling data; and a structure parameter error value of the to-be-calibrated machine and an eccentricity parameter of an angle encoder are obtained by identification of a sequential quadratic programming algorithm.

The invention provides a parameter identification method of a mechanical arm. The parameter identification method of the mechanical arm comprises the steps of defining an M-DH parameter, and writing out a homogeneous transformation matrix between joint coordinate systems; acquiring each theoretical joint angle value, substituting into a robot kinematics forward analysis model, and solving to obtain a theoretical terminal position and an error expression; measuring a practical terminal position, and solving an error value between the theoretical terminal position and the practical terminal position; measuring terminal positions of multiple groups of joint points, building a system of linear equations, and utilizing a least squares algorithm for solving an error value of the M-DH parameter;correcting the M-DH parameter, and utilizing the corrected M-DH parameter to obtain a corrected terminal position; and judging whether an error between the corrected terminal position and the practical terminal position is converged or not, and if yes, finishing the flow. The invention further provides a parameter measurement method and a parameter measurement device of the mechanical arm, a parameter identification device of the mechanical arm, a terminal device and a storage medium. By utilizing the embodiment of the invention, the parameter identification accuracy can be improved.

The invention discloses a high-accuracy calibration method of a joint coordinate measuring machine, which comprises the following steps that the measuring machine acquires a plurality of pairs of taper holes in each calibration plate with different poses for many times; each pose corresponds to a group of joint angles and a measuring head coordinate; a plurality of groups of joint angle vectors and measuring head coordinates are obtained; a plurality of groups of two-point distance values between the taper holes in the calibration plates are calculated; a standard deviation formula of x, y and z coordinates of each measuring point is obtained; standard deviations of the measuring points are summed; the two-point distance value Li of each calibration plate at the measuring point i subtracts a point distance truth value Lr; standard deviations are obtained and summed; delta xyz and delta L are summed to serve as an objective function of a nonlinear quadratic programming problem; and a structure parameter error value of the joint coordinate measuring machine is obtained by a sequential quadratic programming algorithm.

The invention discloses a walking posture control method of a humanoid biped robot based on a genetic algorithm. A mathematical model of a biped robot is established based on the ZMP control theory and a pose matrix, and the basic gait of a human is simulated. V, height H, arm swing R, body center up and down swing C, body center left and right swing A These six parameters plan the robot's gait, and select optimized T, V, H, R through genetic algorithm The six parameters of , C, and A are brought into the inverse operation of the pose matrix to obtain the values ​​of other joint coordinates of the robot, and solve the method of generating data that stably controls the gait of the robot. It overcomes the problem of large amount of calculation and low efficiency in inverse operation of pose matrix, improves the real-time response ability of the system, and uses the data generated by this method in the humanoid biped robot system to achieve a good control effect.

The invention discloses a robot tool central point calibration method using a two-dimensional measurement functional tablet. A robot is controlled to enable a TCP to make contact with a measurement area of a calibration tool more than three times in different postures in the calibration process, joint coordinate information of the robot is recorded in the contact process, and the two-dimensional coordinates of the contact points in a measurement plane are output through the two-dimensional position measurement function of the calibration tool; the coordinates of the TCP in a robot tail end tool coordinate system are calculated according to measured joint rotation angle information of the robot and the coordinate data of the contact points on the tablet and in combination with the structural parameters of the robot, and the calibration accuracy is evaluated. According to the calibration method, the TCP of the robot does not need to coincide with the same fixed point multiple times, the point-point coincidence requirement is weakened to point-surface coincidence, and operation is easy. When the TCP of the robot has small deviation, autonomous calibration can be realized.

The invention provides a PCBA solder joint inspection method based on machine vision. The solder joint position and quality are judged by virtue of image acquisition and analysis. The method specifically comprises the following steps: S1, presetting detection parameters, wherein the detection parameters comprise the solder joint standard coordinates and solder joint standard parameters which are in one-to-one correspondence; S2, photographing to-be-detected PCBA, thereby obtaining a PCBA image; S3, analyzing the PCBA image, and extracting the coordinate and surface parameters of the to-be-detected solder joint; S4, comparing the coordinates of the detected solder joint with the solder joint standard coordinates to and judge whether the solder joint position is accurate; and S5, comparing the solder joint surface parameters with the solder joint standard parameters according to the solder joint coordinates and the solder joint standard coordinates to judge whether the solder joint quality is qualified. According to the method disclosed by the invention, the solder joint detection is performed by utilizing the machine vision technology, the problems that the solder joints are high in intensity and difficult to distinguish, detection leakage is easily caused and the like are effectively solved, the solder joint detection accuracy is guaranteed, the detection efficiency is improved, and the labor cost is reduced.

The invention relates to a robot original point demarcating method and device, a storage medium and computer equipment. The method comprises the following steps that initial joint coordinates corresponding to each joint axis initial state of a robot are obtained, according to the initial joint coordinates, preset reference points at the terminal of the robot in various different attitude trends and current initial joint coordinates corresponding to the joint axis initial states when the preset reference points are coincide are obtained, according to the current joint coordinates, position coordinates corresponding to each attitude are obtained, distance deviation of the every two position coordinates are calculated, when any distance deviation is higher than a preset value, the initial joint coordinates are updated by using a search algorithm, according to the updated initial joint coordinates, the current joint coordinates are updated, the step that according to the current joint coordinates, position coordinates corresponding to each attitude are obtained is entered, and until each distance deviation is less than the preset value, the corresponding target initial joint coordinates are obtained to be used as an original point of the robot. By means of the robot original point demarcating method, the cost can be reduced, and the efficiency can be improved.

The invention discloses a virtual upper limb control system based on myoelectricity and motion capture and a method of the system. The system comprises a myoelectric device, a motion capture device, avirtual game development environment and a virtual / augmented reality device, wherein the virtual game development environment comprises a virtual upper limb model and a virtual game scene; the myoelectric device is used for detecting myoelectric signals and gesture information of different gestures and sending the myoelectric signals and the gesture information to the virtual upper limb model; the motion capture device is used for acquiring joint coordinates and spatial posture information of upper limbs and sending the joint coordinates and the spatial posture information to the virtual upper limb model. The virtual game scene receives data sent by the virtual upper limb model and sends the data to the virtual / augmented reality device, the virtual / augmented reality device feeds back visual auditory stimulation to the user, and virtual objects are placed in a corresponding real environment for operation. The system has the advantages of being low in cost, high in precision, comfort degree, timeliness and interactivity and diversified in scene, has an upper limb integrated game pattern and can increase the game enthusiasm of users.

The invention provides a user identity recognition method and system combined with user gait information, and the method comprises the steps: carrying out the posture detection of each frame of pedestrian object in a video sequence of an original data set through a two-dimensional posture estimation system, and extracting the posture information; preprocessing the extracted joint coordinate sequence to generate a human skeleton data set; and finally, constructing a space-time diagram convolutional network model, dividing the skeleton diagram into six sub-diagrams, sharing joints among the sixsub-diagrams, learning an identification model by using the diagram convolutional network, training by using the constructed data set, and optimizing network parameters by using a multi-loss strategycombining classification loss and comparison loss and random gradient descent. And predicting the accuracy of the trained model by using the verification set. According to the method, effective information of the joint points is fully utilized, the motion state in the time dimension is reserved as much as possible, high robustness is achieved for clothes changes and carrying states, and good generalization capacity is achieved on a cross-view task.

A skeleton-based behavior detection method The invention relates to a terminal device and a computer storage medium, which relate to a limb motion detection technology in the biological detection field, wherein the method comprises the following steps: acquiring original image data; determining a skeleton joint node in the original image data based on an image detection algorithm, and acquiring the joint node coordinates of the skeleton joint node; Converting the joint coordinates based on a preset conversion rule to obtain a joint feature vector; Classifying and recognizing the original imagedata based on the image classification algorithm and the joint feature vector to obtain a behavior detection result can improve the accuracy and robustness of skeleton behavior detection.

The invention relates to a gait training mechanical leg which comprises a treadmill and two single-leg simulators. The single-leg simulators are parallelly disposed on a frame in bilateral symmetry. The frame is fixedly connected with the treadmill, so that training shoes at the lower ends of the single-leg simulators are allowed to step on a footboard of the treadmill. Three drivers are connected to hip joints, knee joints and ankle joints respectively in a crossing manner. The three drivers are the same in structure. The three joints coordinate to rotate to allow for complete human gait training so as to simulate motion tracks of the hip joints, the knee joints and the ankle joints during walking. The gait training mechanical leg is capable of effectively simulating the motion tracks of the hip joints, the knee joints and the ankle joints during human gait training, and is compact in structure, light in weight, stable in performance and easy to control.

The invention relates to a human upper body movement recognition method based on key frames and random forest regression, which is characterized in that: key joint coordinates of the human upper bodyare simultaneously obtained by using OptiTrack and its supporting software OptiTrack and Kinect v2; the key joint coordinates of the human upper body are simultaneously obtained by using OptiTrack andKinect v2. The regression function of the eigenvalue difference between the key frames from Kinect and OptiTrack is studied by using the method of stochastic forest regression, and two functions arerealized: (1) inputting the frame from Kinect, predict the eigenvalue difference from the stochastic forest, and then correcting the skeleton; (2) inputting the frame obtained from Kinect, and judgingthe action by the random forest prediction gesture mark. The method can make up for the shortcomings of the existing Kinect-based upper body motion recognition methods, and solve the problem that only simple motion can be recognized and the recognition accuracy is not high.

The invention belongs to the field of industrial robots, and discloses a method for industrial robot programming route diagnosis and speed optimization. The method comprises the following steps that (1) an original programming code of the moving trail of an industrial robot is obtained, an industrial robot model is built, and an attribute parameter of the industrial robot is obtained; (2) the moving trail is discretized, line segments with the same arc length are formed, and a series of discrete points are obtained; (3) joint coordinates of the discrete points are inversely obtained; (4) a joint speed, an accelerated speed and a moment of force of each discrete point are calculated; (5) areas which are not up to the programming speed are diagnosed and are subjected to speed optimization; and (6) the programming code after speed optimization is output, wherein compared with the original programming code, programming points are added and programming speed is optimized. The speed optimization of the programming trails is achieved, and therefore the programming code satisfying robot dynamics is obtained. When the programming code is in computer operation, running time of the robot is shortened, and running of the robot is stabler.

The embodiment of the invention discloses a motion information recognition method, and the method comprises the steps: receiving joint coordinate data, which is transmitted by motion capturing equipment and captured in a preset time period, of a to-be-recognized object; calculating and obtaining the joint inclined angle data of the to-be-recognized object according to the joint coordinate data of the to-be-recognized object; obtaining the change trend information of the to-be-recognized object in the preset time period according to the joint inclined angle data; and querying the motion information matched with the change trend information from a preset motion database. The invention also discloses an embodiment of a motion information recognition system. According to the embodiment of the invention, the method and system improve the motion information recognition timeliness.

The invention discloses a system and method for calibrating the tool center point of a robot. The system may include a first image sensor, a second image sensor, and a controller. The first image sensor may have a first image central axis. The second image sensor may have a second image central axis not parallel to the first image central axis, and intersect the first image central axis at an intersection point. The controller may control a robot to repeatedly move a tool center point thereof between the first and the second image central axis. The controller may record a calibration point including the coordinates of the joints of the robot when the tool center point overlaps the intersection point, and then move the tool center point and repeat the above steps to generate several calibration points, whereby the controller may calculate the coordinate of the tool center point according to the calibration points.

The invention relates to a solder joint positioning method and a solder joint positioning device. The solder joint positioning method comprises the steps of acquiring a first image of a workpiece to be welded, and extracting identifying information of the workpiece to be welded according to the first image; according to the identifying information of the workpiece to be welded and a preset corresponding relation, acquiring second solder joint coordinates and second characteristic point coordinates corresponding to the identifying information; acquiring first characteristic point coordinates onthe first image according to the first image; correcting the second solder joint coordinates according to the first characteristic point coordinates and the second characteristic point coordinates soas to obtain first solder joint coordinates; and identifying a solder joint of the workpiece to be welded according to the first solder joint coordinates. According to the solder joint positioning method, the first image of the workpiece to be welded is acquired through using a vision positioning mechanism in advance, and the solder joint coordinates on the first image can be acquired, so that abasis is provided for adjusting the solder joint coordinates through a welding mechanism in the follow-up process. By adopting the vision positioning mechanism, accurate positioning can be realized, and the reliability of solder joint positioning is improved.

The invention discloses a man-machine collaboration robot grabbing system and a work method thereof. The man-machine collaboration robot grabbing system comprises a wearing type inertial sensor module, a camera spacing module, a Gaussian model EM algorithm training module, a Gaussian model processing module and an adaptive robot joint coordinate extracting module. In the man-machine collaborationapplication site, the man-machine collaboration robot grabbing system does not need to carry out calibration and kinematics inversion of a robot vision system, the professional skills required of an operator can be reduced, in addition, the man-machine collaboration robot grabbing system can achieve mapping without a large amount of samples, and the robot track is smooth.

The invention discloses a virtual digital sculpture method based on a natural gesture. The virtual digital sculpture method comprises the following steps of obtaining a gesture position through Leap Motion; and performing virtual sculpture modeling. According to the digital sculpture method, two major models, including an established to-be-sculptured model and a virtual hand model which is established based on a joint coordinate obtained from Leap Motion, are adopted in a task of the virtual sculpture; when a human hand moves under Leap Motion to further control a virtual hand and a fixed to-be-sculptured model to be subjected to slight collision, whether a crossed point exists or not in the collision position is detected through a triangle-triangle crossing algorithm in a three-dimensional space; and then real-time correction on a crossed point coordinate is performed through a virtual sculpture deformation algorithm in an intersecting line position of the crossed point according to calculation of the collision depth, so as to achieve a sculpture deformation effect.

The invention discloses a tumble detection method based on human body posture estimation. The method belongs to the technical field of artificial intelligence, and comprises the following steps: estimating the posture of a person appearing in a camera by using an open-source posture estimation detection algorithm, and then further finding out coordinates of seven parts of a chest, two hips, two knee joints and two ankle joints of a marked person from the result of the detection algorithm; representing the trunk and legs of a person by using connecting lines of the corresponding body part points and the joint coordinate points; judging whether the human body falls down or not by analyzing slope information of the trunk and the legs in the figure. The method can be applied to a public scene,and can timely and effectively discover accidental tumbling, tumbling and other conditions through the monitoring camera.

A pair of manipulators (10A, 10B) are caused to take a plurality of attitudes in a state where distal ends of the manipulators (10A, 10B) are coupled to each other, coordinates of joints between links at each attitude change are acquired on the basis of detection signals, at each attitude change, of rotary encoders provided for servomotors that drive the links of the manipulators (10A, 10B), and a position and attitude of an installation point (WB) of a slave robot with reference to an installation point (WA) of a master robot are calculated on the basis of the joint coordinates acquired at the corresponding attitude change in a forward kinematics manner. A deviation vector for each attitude change between actual measured values of the installation point (WB) of the slave robot and the calculated values of the installation point (WB) of the slave robot is calculated, and robot constants of both manipulators are identified from the deviation vector.

A method and apparatus for detecting a garbage dumping action in real time on a video surveillance system are provided. A change region, which is a motion region, from an input image is detected, joint information including joint coordinates corresponding to a region in which joints exist is generated, and an object held by a person from the image using the change region and the joint information is detected. Then, an action of dumping the object based on a distance between the object and the joint coordinates is detected.