Unlock instant, AI-driven research and patent intelligence for your innovation.

Multi-pedestrian three-dimensional tracking method and system based on monocular vision

A technology of 3D tracking and monocular vision, which is applied in the fields of 3D positioning and 3D tracking, artificial intelligence monitoring, and pedestrian detection in artificial intelligence monitoring. It is difficult for the camera to obtain depth of field information, etc., to improve the accuracy of 3D tracking, improve computing efficiency, and improve operating efficiency

Pending Publication Date: 2022-01-04
RENMIN UNIVERSITY OF CHINA
View PDF0 Cites 0 Cited by
  • Summary
  • Abstract
  • Description
  • Claims
  • Application Information

AI Technical Summary

Problems solved by technology

However, the existing monocular vision-based intelligent monitoring system has at least the following two problems: 1. Because the monocular camera is difficult to obtain field depth information, the existing monocular camera is mainly used for 2D target detection and 2D positioning, and lacks three-dimensional space An effective means of positioning and tracking; 2. The existing intelligent monitoring system is mainly a single-design solution for multi-pedestrian detection, re-identification, positioning and tracking, and there is still a lack of efficient, end-to-end based on monocular camera Video streaming, an effective method for real-time calculation and tracking of multi-pedestrian 3D spatial positions
Bertoni et al. (Monoloco: Monocular 3d pedestrian localization and uncertainty estimation, Bertoni et al., Proceedings of IEEE International Conference on ComputerVision, 6860-6870, 2019) predict the spatial position of pedestrians through attitude key points, and give pedestrians through uncertainty estimation The interval range of the position, but this method needs to use another independent neural network to predict the key points of the pedestrian pose from the monocular image

Method used

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
View more

Image

Smart Image Click on the blue labels to locate them in the text.
Viewing Examples
Smart Image
  • Multi-pedestrian three-dimensional tracking method and system based on monocular vision
  • Multi-pedestrian three-dimensional tracking method and system based on monocular vision
  • Multi-pedestrian three-dimensional tracking method and system based on monocular vision

Examples

Experimental program
Comparison scheme
Effect test

Embodiment 1

[0030] This embodiment discloses a multi-pedestrian three-dimensional tracking method based on monocular vision, such as figure 1 , 2 , 3, including the following steps:

[0031] S1 performs feature extraction on the training image set obtained by the monocular camera.

[0032] The method of extracting features from the training image is as follows: mark the category k of the object j in the training image set, and the pixel coordinates of the upper left corner of the bounding box of the object j Pixel coordinates of the lower right corner Create object class labels. Input a fixed-size image (such as W×H×3) into the convolutional neural network, extract features through convolution operations, and obtain a feature map F with a size of W / S×H / S, where W is the object j’s Width, H is the height of object j, and S is the step size of downsampling.

[0033] S2 simultaneously performs pedestrian detection and corresponding pedestrian ID embedding learning according to the ext...

Embodiment 2

[0071] Based on the same inventive concept, this embodiment discloses a multi-pedestrian three-dimensional tracking system based on monocular vision, including:

[0072] The feature extraction module is used to extract features from the training image set obtained by the monocular camera;

[0073] The pedestrian detection and ID learning module is used to simultaneously perform pedestrian detection and corresponding pedestrian ID embedding learning according to the extracted features; the 3D positioning module is used to perform 3D positioning of pedestrians according to the pedestrian detection results;

[0074] The re-identification module is used to combine the ID learning results to re-identify the pedestrian detection results;

[0075] The pedestrian three-dimensional tracking module is used to determine the movement trajectory of each pedestrian according to the re-identification result and the position of the pedestrian, so as to track it.

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to View More

PUM

No PUM Login to View More

Abstract

The invention belongs to the technical field of artificial intelligence monitoring, and relates to a multi-pedestrian three-dimensional tracking method and system based on monocular vision. The method comprises the following steps: S1, performing feature extraction on a training image set obtained by a monocular camera; S2, performing pedestrian detection and ID embedding learning of corresponding pedestrians at the same time according to the extracted features; S3, performing 3D positioning on the pedestrian according to the pedestrian detection result; S4, re-identifying the pedestrians in combination with an ID embedded learning result; and S5, according to the re-identification result and the 3D positions of the pedestrians, determining the motion trails of the pedestrians, and carrying out three-dimensional tracking on the pedestrians. The method can accurately detect and associate the same pedestrian, accurately predict the 3D position of the pedestrian and draw the 3D motion track of the pedestrian, can achieve real-time operation efficiency, and improves the operation efficiency of the whole model on the basis of ensuring that the tracking accuracy is competitive.

Description

technical field [0001] The invention relates to a three-dimensional tracking method for multiple pedestrians based on monocular vision, which belongs to the technical field of artificial intelligence monitoring, in particular to the technical fields of pedestrian detection, three-dimensional positioning and three-dimensional tracking in artificial intelligence monitoring. Background technique [0002] With the rapid development of the Internet of Things and artificial intelligence technology, camera-based security monitoring, ecological monitoring, and smart homes have been widely used. In these applications, the main target of monitoring is pedestrians. Because monocular cameras have the advantages of low cost and ease of use, multi-pedestrian detection, re-identification, positioning and tracking based on monocular vision has become a research hotspot in this field. However, the existing monocular vision-based intelligent monitoring system has at least the following two pr...

Claims

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to View More

Application Information

Patent Timeline
no application Login to View More
IPC IPC(8): G06T7/246G06T7/73G06T7/277G06K9/62
CPCG06T7/246G06T7/73G06T7/277G06T2207/30196G06T2207/20081G06T2207/20084G06T2207/30241G06F18/22
Inventor 王永才孙宏宇
Owner RENMIN UNIVERSITY OF CHINA