68 results about "Pose estimation algorithm" patented technology

The present invention belongs to the field of computer-aided human face image re-construction technology. The present invention features that human face as model establishing object is first photographed with two, one upper and one lower, stere ovideo cameras to obtain video sequence from the front to the side of the face with unchanged expression; the precise posture parameters in different times are then obtained via autoamtic posture estimating algorithm; after the initiation of model and posture, 3D information are extracted separately from two types of 2D threads including one stereo reconstruction type of the image and the other matching and correcting type of model projection contour and image contour; and the 3D face model is finally established.

The invention discloses a multi-modal action recognition method based on a depth neural network. The method comprehensively utilizes multi-modal information such as video images, optical flow diagramsand human skeleton. The specific steps are as follows: firstly, a series of preprocessing and compression are performed on the video; Obtaining an optical flow graph based on adjacent frames of the video; The human skeleton is obtained from the video frame by using the attitude estimation algorithm, and the path integral characteristics of the skeleton sequence are calculated. The obtained optical flow graph, skeleton path integration feature and original video image are input into a depth neural network with multi-branch structure to learn the abstract spatio-temporal representation of humanmotion and correctly judge its motion category. In addition, the pooling layer based on attention mechanism is connected to the video image branch, which enhances the abstract features closely related to the final motion classification results and reduces the irrelevant interference. The invention comprehensively utilizes multimodal information, and has the advantages of strong robustness and high recognition rate.

The present invention provides a human movement identification method through fusion of a deep neural network model and binary system Hash, belonging to the technical field of mode identification. Themethod comprises the steps of: performing preprocessing of a movement identification database, dividing the movement identification database into frame sequences, calculating an optical flow graph, employing an attitude estimation algorithm to calculate coordinates of human joint points, and employing result coordinates to extract video area frames; employing a pre-training VGC-16 network model to extract FC (Full-Convolutional) features of RGB flows and optical flows of the videos, selecting key frames from the video frame sequences, and obtaining a difference of the FC features corresponding to the key frames; performing binary processing of the difference; employing a binary-hashing method to obtain uniform feature expression of each video; employing a plurality of normalization methods such as L1 and L2 to obtain feature expressions of the videos after the fusion of the uniform feature expressions and the PCNN features; and finally, employing a support vector machine algorithm totrain a classifiers to identify the human movement videos. The human movement identification method through fusion of the deep neural network model and the binary system Hash has a high movement identification correct rate.

The embodiment of the invention discloses a depth estimation method and apparatus of a monocular video, a terminal, and a storage medium. The method includes steps: obtaining an image frame sequence of the monocular video, and calculating an attitude relation between each two adjacent image frames in the sequence according to a camera attitude estimation algorithm; regarding the image frames in the sequence as input of a preset neural network model in sequence, and determining an initial depth map and an initial uncertainty distribution map of each image frame according to output of the presetneural network model; performing inter-frame information transmission and fusion according to the attitude relations and the initial depth maps and the initial uncertainty distribution maps of the image frames, and determining final depth maps and final uncertainty distribution maps of the image frames in sequence. According to the technical scheme, depth restoration of the image frames of the monocular video can be performed, the prediction precision of the depth maps can be improved, and the uncertainty distribution of the depth maps can be obtained.

The invention discloses a coarse-to-fine video target behavior identification method, and the method comprises the steps: firstly obtaining human body key points by using an attitude estimation algorithm or annotation information in a video, and cutting and zooming different body parts of a human body; taking the deep neural network as a feature extraction network, and extracting feature vectors of different part areas; and iteratively training a classifier by utilizing the extracted feature vectors of different parts, and searching for the optimal coarse classification of the behavior; for the coarse classifier and each fine-grained classifier, selecting different parts to be cascaded with the global feature vector; carrying out individual training of the classifiers; and carrying out probability fusion on classification results of the coarse-grained classifier and the fine-grained classifier to obtain a final behavior recognition result. According to the method, a behavior recognition framework from coarse to fine is constructed, cascading is utilized to train classifiers in a targeted mode for feature expressions of different body parts of a person with different granularities,and therefore the probability of wrongly dividing similar behaviors is effectively reduced, and the overall behavior recognition accuracy is improved.

The invention discloses a construction safety helmet wearing monitoring method based on computer vision human body posture estimation. The construction safety helmet wearing monitoring method comprises the following steps: S100, detecting and outputting body skeleton postures of all individuals in an image according to a real-time multi-person posture estimation algorithm model OpenPose; S200, detecting a worker and a safety helmet in the image according to a deep learning target detection algorithm model YOLOv3, and using two types of rectangular frames for identification and output respectively; and S300, integrating the two algorithm models, and judging whether a worker wears the safety helmet or not in the figure according to the intersection surface ratio of the two types of rectangular frames and the comparison of the diagonal length of the rectangular frames of the safety helmet and the length from the center point of the face of the person to the center point of the safety helmet. The construction safety helmet wearing monitoring method can accurately judge whether the safety helmet is correctly worn or not, is clear in logic process and high in detection speed, and has important significance in guaranteeing effective wearing of the safety helmet on a construction site in real time and maintaining the life safety of constructors.

The invention discloses a virtual person-based dance teaching data processing method and system. A virtual person runs in an intelligent device and has a preset image feature and a preset attribute; and voice, emotion, vision and perception abilities are enabled in an interactive state. The method comprises the steps of collecting and analyzing human body multimode data, and extracting dance videoclips from the data; performing deconstruction and estimation on dance actions in the dance video clips based on a human body pose estimation algorithm to generate a human body 3D skeleton action sequence corresponding to the dance actions; rendering the human body 3D skeleton action sequence to obtain a rendering result; and displaying the rendering result of the human body 3D skeleton action sequence by utilizing the virtual person. According to the method and the system, the virtual person can be used for performing fluent and natural dance action display, so that the user experience of the virtual person is greatly improved.

The invention discloses a monocular vision mileage calculating device. The monocular vision mileage calculating device comprises a feature detection and correlation module 1 and a pose estimation module 2, wherein the feature detection and correlation module 1 is used for detecting GFtT feature points at each moment when a new image is obtained, and tracking positions of the GFtT feature points in the image at the next moment so as to obtain a GFtT feature point correlation set in a certain time and outputting the GFtT feature point correlation set to the pose estimation module; the pose estimation module 2 is used for modeling by a pose estimation algorithm based on constraint of a vehicle dynamic model according to the GFtT feature point correlation set output by the feature detection and correlation module so as to obtain a relative pose transformation between adjacent moments. The monocular vision mileage calculating device has relatively high efficiency and accuracy, can meet real-time positioning and navigation requirements of an intelligent vehicle, can be used in a complex environment with a large quantity of movement obstacles, is free of assumed limit in static scenes and is capable of meeting the use requirements of an intelligent vehicle in a complex urban road environment.

The invention provides a non-texture spatial object attitude estimation algorithm based on contour point ORB (Oriented FAST and Rotated BRIEF) feature matching, which belongs to the technical field ofdigital image processing. On the basis of three-dimensional model prior, data information contained in the contour of a projected image is made full use of, ORB feature matching and color indexing are used to build a 2D-3D feature corresponding relationship from an input image to an object three-dimensional model, matching errors are used to build a confidence probability matrix, and a weighted orthogonal projection algorithm is put forward to calculate six-degree of freedom attitude parameters of the non-texture spatial object. The ORB features improve the accuracy of contour point matching,and certain robustness is realized in a condition of large offset of an initial attitude relative to a real attitude. The contact between building of the 2D-3D corresponding relationship subproblem and calculation of the attitude parameter subproblem is thoroughly explored, the matching errors are used to build the confidence probability matrix as the prior information for calculating the attitude parameters, the phenomenon that an RANSAC algorithm is used to eliminate error matching points is avoided, and the algorithm calculation efficiency and the precision are improved.

The invention discloses a six-degree-of-freedom pose estimation algorithm based on projection point coordinate regression, Based on the single-step target detection algorithm, this paper extends the BB8 algorithm to solve the problem of pose estimation. Using the BB8 algorithm for reference, the six-degree-of-freedom pose parameters are calculated by EPnP algorithm for the projection coordinates of the vertices of the three-dimensional boundary box for every object of interest detected. The convolution neural network provided by the invention can carry out end-to-end training and prediction for the pose estimation target, the computational efficiency and the precision of pose estimation are improved, In the multi-object scene, only one EPnP calculation is needed for each object instance detected, and the algorithm does not need to run repeatedly for many times, thus the running time of the algorithm is saved. The algorithm can achieve the current optimal precision of pose estimation without post-processing of pose correction, and can realize real-time processing.

The invention discloses an examination cheating detection method based on postures. The method comprises: through a posture estimation method, acquiring a human body posture sequence in the video; using a sliding window method for detecting characteristics and changes of postures in the window, analyzing the posture state, judging whether a cheating action is a suspected cheating action or not according to the characteristics of the cheating characteristics, further capturing a hand image of the cheating action, and judging whether a paper slip or a mobile phone or other cheating tools exist on the hand or not through an image detection method. The posture of an examinee is obtained by using the posture estimation algorithm. The state of the examinee is recognized through kinematic analysis on a human skeleton sequence, reliability is higher, and the examinee is not affected by the environment of an examination room. Cheating behaviors are detected from human body posture motion data,whether the cheating suspicion exists or not is judged according to posture data changes in space and time, and the detection method better conforms to the thinking of people and is more accurate, more visual and more specific.

The invention discloses a six-degree-of-freedom pose estimation algorithm based on bounding box outer key point positioning, and the algorithm comprises the steps: inputting a three-dimensional modeland an RGB image, obtaining the size information of a target in the image in three dimensions, and determining the coordinates of eight vertexes of a target three-dimensional bounding box; detecting the RGB image through a convolutional neural network to obtain five feature maps with different scales, a target category of each position and a two-dimensional bounding box prediction result; carryingout non-maximum suppression operation to obtain the category of the target instance in the redundancy-removed image and a two-dimensional bounding box prediction result; positioning the key points inthe image to obtain positions and positioning confidence coefficients of the eight key points; and calculating the six-degree-of-freedom pose of the target relative to the camera by using an EPnP algorithm. The method can effectively improve the positioning precision of the key points outside the bounding box, and improves the pose estimation precision while guaranteeing the real-time processingcapability.

Provided are a method, medium, and apparatus for docking a mobile robot. The method includes: (a) receiving a docking signal, which is transmitted from a signal transmitting unit, by using a signal receiving unit loaded in the mobile robot while the mobile robot is moving; (b) estimating a range of poses of the signal transmitting unit based on the received docking signal; (c) updating estimated poses of the signal transmitting unit by reflecting the estimated range of poses in a pose estimation algorithm when the pose of the signal transmitting unit, which was estimated using the pose estimation algorithm, exists within the estimated range of poses and adding a new pose of the signal transmitting unit, which is to be estimated using the pose estimation algorithm by reflecting the estimated range of poses, when the pose of the signal transmitting unit, which was estimated using the pose estimation algorithm, does not exist within the estimated range of poses; and (d) attempting to return the mobile robot to a charging station by using a plurality of poses of the signal transmitting unit, which were estimated by repeating operations (a) through (c) and using the pose estimation algorithm, in order of accuracy, until the mobile robot is docked in the charging station.

The invention relates to a human hand three-dimensional posture estimation method and device based on three-dimensional point cloud which mainly solves the problem of how to recover the three-dimensional posture of a human hand from the human hand point cloud obtained by a single depth map, and has the main technical difficulties of disordered point cloud arrangement, higher noise, rich gesture changes of the human hand, self-shielding of the human hand caused by a shooting angle and the like. The invention provides a human hand posture estimation algorithm based on a deep neural network by which the features can be adaptively extracted from the rich training data. Meanwhile, the local and global features of the point cloud can be predicted while the three-dimensional positions of the joint points of the human hand are regression in real time, the generalization ability of the network is improved through the internal connection of joint labeling, and the problem that the generalizationability of the features extracted by a single-task network is poor, is solved. The actual use verifies that the method and the device have the advantages of high automation degree, high precision andreal-time performance, and can meet the professional or popular application requirements.

The invention discloses a human body action recognition method in monitoring based on deep learning and posture estimation. The method comprises the following steps: constructing a multi-flow action recognition model based on relative part joint feature representation; preprocessing human body skeleton action data and converting relative part joint feature representation; inputting the converted relative part joint feature representation into a multi-flow identification model for model training and evaluation, and selecting an optimal model with the highest convergence identification rate after multiple rounds of iteration; obtaining a monitoring segment in a real-time scene of the monitoring video, obtaining a skeleton action sequence of a human body in the monitoring segment by adoptingan attitude estimation algorithm, and preprocessing the skeleton action sequence; performing feature representation conversion on the preprocessed skeleton action sequence; using the optimal model toidentify human body actions in the bone action sequence after preprocessing and feature representation conversion, and obtaining an action classification result; and comparing the identified classification result with a preset dangerous action category, and returning a comparison result to the monitoring worker.

The invention belongs to the field of spacecraft navigation, and particularly relates to a deep learning and geometric algorithm combined non-cooperative target relative pose estimation method, whichcomprises the steps: manually selecting 2D key points by utilizing a non-cooperative target data set, and reconstructing 3D model combination of a non-cooperative target by combining multi-view triangulation; re-projecting the 3D coordinates to an image plane to obtain 2D coordinate estimated values of the key points; designing a target detection and semantic segmentation network and a key point regression network, and performing supervised regression on 2D key point coordinates; and minimizing errors of 2D-3D corresponding point coordinates by using a nonlinear least square method to estimatea six-degree-of-freedom relative position and a relative attitude of the non-cooperative target. According to the method, deep learning and a geometric optimization algorithm are effectively combined, research of a high-precision and high-speed pose estimation algorithm based on a visual image can be realized, and the method is an innovative application of deep learning in the aerospace field.

The invention provides a gesture recognition method and device based on graph convolutional neural network. The method comprises the following steps: obtaining a gesture joint point time-space diagramby using an attitude estimation algorithm, and performing normalization processing on the gesture joint point time-space diagram to obtain data to be calculated, so that data to be calculated can becalculated through an established time-space diagram convolutional neural network-gesture recognition model, and finally, acquiring a recognition result. The calculated data is classified through sixspace-time convolution units, three pooling layers and a support vector machine classifier, thereby improving real-time performance of gesture recognition while improving accuracy of gesture recognition.

The invention belongs to the technical field of artificial intelligence, and discloses a behavior recognition method based on a local articulation point trajectory space-time volume in a skeleton sequence, and the method comprises the steps: extracting the local articulation point trajectory space-time volume from inputted RGB video data and skeleton articulation point data; extracting image features by using a pre-training model based on the RGB video data set; constructing a codebook for each different feature of each joint point in the training set and encoding the codebook, and connectingthe features of the n joint points in series to form a feature vector; and performing behavior classification and recognition by using an SVM classifier. According to the method, manual features and deep learning features are fused, local features are extracted by using a deep learning method, and fusion of multiple features can achieve a stable and accurate recognition rate. According to the invention, the 2D human body skeleton estimated by the attitude estimation algorithm and the RGB video sequence are used to extract the features, the cost is low, the precision is high, and the method hasimportant significance when applied to a real scene.

The invention discloses a constructor dressing standardization detection method and device based on visual analysis, and the method comprises the steps: collecting an image sample of a constructor, capturing a local image of the head of the constructor, and dividing the local image into a safety belt wearing mode and a safety helmet non-wearing mode; training a classification model on the sample set by utilizing ResNet to obtain a helmet wearing classification model; acquiring a real-time video stream of a construction site, and extracting to obtain a key frame image; analyzing the key frame image by utilizing an OpenPose human body posture estimation algorithm, judging whether a human body key point is detected or not, and if so, positioning a head area and an arm area; otherwise, judgingwhether the short-sleeved shirt is worn or not in the arm area by adopting a skin color model; calling a safety helmet wearing classification model to classify and identify the head area, and judgingwhether a constructor wears the safety helmet or not. According to the invention, 24-hour all-weather nonstandard behavior detection of constructors can be realized, so that the safety supervision ofa construction site is realized, and the construction efficiency and safety are improved.

The invention provides an object grabbing method and device. The method comprises the steps: acquiring image data of a to-be-grabbed object, Wherein the image data comprises three-dimensional coordinate information of each point on the surface of the to-be-captured object; Generating at least one grabbing mode according to the image data of the to-be-grabbed object, Wherein the grabbing mode comprises three-dimensional coordinate information of at least one grabbing point and a grabbing angle; Inputting the image data and the grabbing mode of the to-be-grabbed object into a grabbing evaluationmodel, and obtaining an evaluation score of the grabbing mode; Selecting a grabbing mode with the evaluation score meeting the rule to grab the to-be-grabbed object; Therefore, a posture estimation algorithm with low robustness is avoided, the trained grabbing evaluation model is adopted, the grabbing mode with the evaluation score meeting the rule is obtained for grabbing operation, and therefore the accuracy of the grabbing mode is improved, and the success rate of object grabbing is increased.

The invention discloses a hybrid deep regression network-based head pose estimation method and belongs to the computer vision and machine learning field. According the method of the invention, on thebasis of a traditional regression algorithm and a deep network framework, image information of different modalities is applied to a head pose estimation problem for the first time; and sub-networks ofa plurality of modalities, which are obtained through training, are fused, so that a final hybrid deep regression network can be obtained. Compared with other head pose estimation methods, the algorithm of the invention can achieve higher estimation accuracy and have high robustness; and in addition, the method of the present invention has a certain universality, and is not only applicable to typical head pose estimation problems such as human-computer interaction, safe driving, and face recognition, but also can expand problem conditions to other deep regression problems.

A method for optimizing fiducial marker and camera positions / orientations is realized to simulate camera and fiducial positions and pose estimation algorithm to find best possible marker / camera placement comprises the steps of: acquiring mesh data representing possible camera positions and feasible orientation boundaries of cameras on the environment of tracked object; acquiring mesh data representing possible active marker positions and feasible orientation placements of markers on a tracked object; pose data representing possible poses of tracked object under working conditions; initializing the control parameter for camera placement; create initial solution strings for camera placement; solving marker placement problem for the current camera placement; evaluating the quality of the current LED and camera placement taking pose coverage, pose accuracy, number of placed markers, number of placed camera etc. into account; determining if a stopping criterion is satisfied.

The present invention relates to the field of computation and simulation and covers methods to optimize the fiducial marker positions in optical object tracking systems, by simulating the visibility. The method for optimizing tracker system which is realized to simulate camera and fiducial positions and pose estimation algorithm parameters to optimize the system comprises the steps of; acquire mesh data representing possible active marker positions and orientations on a tracked object, pose data representing possible poses of tracked object, camera positions and orientations; compute visibility of each node from all camera viewports and generate a visibility value list; select the node with highest visibility count as a marker placement node; remove nodes closer to the selected node than a threshold; remove the pose(s) having a predetermined number of selected nodes; does percentage of all poses have predetermined number of selected nodes?; project selected node positions on the image plane of each camera viewport and calculate the pose of the mesh using the tracker algorithm to be optimized; calculate pose error and pose coverage by comparing algorithm results with initial data; record and output results; and select among the results a parameter set satisfying at least one constraint.

The invention relates to a hand raiser detection method based on object detection and attitude estimation. The method comprises the following steps: 1) obtaining a teaching video to be tested; 2) evenly extract picture frames in that teach video to be tested, inputting the trained hand-raising motion detection model, and obtain a picture frame containing the hand-raising motion and a first text file for recording the position of the hand-raising frame; 3) carry out pose estimation on that picture frame containing the hand-raising action to obtain the key point of the human body of all personsin each picture frame and forming a second text file for recording the position of the key point; 4) detect that obtained hand raise according to the first text file and the second text file by usinga heuristic matching strategy. Compared with the prior art, the invention solves the problems of low resolution and motion distortion by improving the attitude estimation algorithm, adopts the heuristic matching strategy to accurately obtain the real hand raiser, and has the advantages of high detection accuracy and detection rate.

The invention provides a six-degree-of-freedom pose estimation system and method based on speckles. The system comprises an infrared emitter, a camera and a computer for algorithm processing. The camera is fixed on a tripod. The infrared emitter can be held by hand, and the visual field of the camera is intersected with an image projected by the infrared emitter during working. The method comprises the following steps: acquiring a reference image projected by an infrared emitter, recovering the complete reference image of the infrared emitter by combining a connected domain algorithm, a Hash algorithm, a voting method, a region growing algorithm, a light beam adjustment method and a panoramic splicing algorithm, and constructing an LUT for the reference image of the infrared emitter; and calculating an intrinsic matrix E by adopting an RANSAC algorithm and a Nister five-point pose estimation algorithm to obtain a pose estimation parameter result. According to the method, the advantagesof active vision are mainly utilized, and an efficient and robust pose estimation result is obtained by combining a connected domain algorithm, a Hash algorithm, a voting method, a region growing algorithm, a light beam adjustment method, a panoramic splicing algorithm, an RANSAC algorithm and a Nister five-point pose estimation algorithm.

The invention relates to an airplane deicing real-time monitoring device comprising an infrared light source, an infrared multi-spectral residual ice detection device, an infrared sensor, a fixed bracket and a main control device, wherein the infrared light source is mounted in the middle of the fixed bracket; the infrared multi-spectral residual ice detection device and the infrared sensor are both mounted at one end of the fixed bracket; the main control device is connected with the infrared light source, the infrared multi-spectral residual ice detection device and the infrared sensor through leads at the same time. Based on the multi-spectral characteristic of the airplane residual ice in a deicing process, the airplane deicing real-time monitoring device provided by the invention determines the relative position of residual ice in a deicing region and automatically identifies the residual ice by adopting a non-contacted infrared sensor and infrared multi-spectral residual ice detection device combined mode and by a method for evaluating the posture of the device relative to the airplane and a coordinate transformation method. The airplane deicing real-time monitoring device can realize automatic identification and positioning of the residual ice in the deicing region and automatic control of a monitoring process, and has the characteristics of high measuring speed, high efficiency and the like.

The invention discloses a construction personnel safety belt wearing detection method and device based on visual analysis, and the method comprises the steps: collecting an image sample of a construction personnel, forming a sample set, capturing a trunk region image of the construction personnel, and dividing the trunk region image into a safety belt wearing state and a safety belt non-wearing state; on the sample set, using a VGG training classification model to obtain a safety belt wearing classification model; acquiring a real-time video stream of a construction site monitoring camera, andextracting to obtain a key frame image; analyzing the key frame image by utilizing an OpenPose human body posture estimation algorithm, judging whether a human body key point is detected or not, andif so, positioning a trunk region of a human body; calling a safety belt wearing classification model to classify and identify the trunk area, and judging whether the constructor wears the safety beltor not. According to the invention, 24-hour all-weather nonstandard behavior detection of constructors can be realized, so that the safety supervision of a construction site is realized, and the construction efficiency and safety are improved.

The invention provides a scanning region positioning device in a residual ice detection process of an airplane. The scanning region positioning device comprises a large-view-field visual positioning sensor, a fixed bracket, a three-degree-of-freedom rotary platform, an infrared multispectral residual ice detection device and a main control device, wherein the fixed bracket is mounted at the upper end of the three-degree-of-freedom rotary platform; the two ends of the fixed bracket are connected with the large-view-field visual positioning sensor and the infrared multispectral residual ice detection device respectively; the main control device is connected with the large-view-field visual positioning sensor, the infrared multispectral residual ice detection device and the three-degree-of-freedom rotary platform. According to the scanning region positioning device, based on inherent geometrical characteristics of the detected airplane, a method for combining the large-view-field visual positioning sensor and the infrared multispectral residual ice detection device is adopted; coordinates of a scanning region and a residual ice position under an airplane body coordinate system can be determined through the device relative to a posture estimation algorithm and a coordinate conversion method of an airplane body, so that automatic positioning of the scanning region and the residual ice position can be realized; a scanning process is controlled automatically; the scanning region positioning device has the characteristics of high measurement speed, high efficiency, high positioning precision and the like.

The invention discloses a 3D body posture estimation algorithm for a single depth image. The method comprises the steps: proposing an improved feature extraction method, and guiding the depth gradientfeature extraction through the comprehensive utilization of the part size information and distance transformation information, thereby greatly improving the expression capability of the extracted features; proposing an error classification processing mechanism-multistage random forest integration algorithm for removing part wrong classified points in order to solve a wrong classification problemin the random forest part classification, and obtaining a more accurate part recognition result; adaptively removing the interference points in the part again through the improved PDA, a position weight threshold processing method and the recognized part size information, and obtaining a more accurate main direction vector; and finally obtaining a posture estimation result through a human body part configuration relation. The method improves the accuracy of a part classification model, can effectively remove the wrong classified interference points in the recognized part, improves the part recognition accuracy, and finally obtains a more accurate 3D body posture estimation result.