Pedestrian coordinate capture device based on image analysis and use method thereof
Through image analysis technology and multi-sensor fusion pedestrian coordinate capture device, the problem that existing equipment cannot accurately capture pedestrian geographical locations is solved, and the accurate location and trajectory of pedestrians is captured and recorded, providing efficient pedestrian information collection and Data upload function.
Patent Information
- Application Number
- CN201910562531.2
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2019-06-26
- Publication Date
- 2024-11-12
- Estimated Expiration
- 2039-06-26
AI Technical Summary
Existing equipment cannot accurately capture pedestrian geographical location information, with large errors and cannot truly reflect pedestrian trajectories, and lacks dedicated sensor devices that actively capture pedestrian geographical location information.
Using a pedestrian coordinate capture device based on image analysis, combined with a mobile object video capturer, processor, gimbal servo mechanism, distance capturer and close-up capturer, the geographic location information of pedestrians is actively captured through image analysis technology and multi-sensor fusion. Movement trajectory and face characteristics, laser rangefinder and electronic compass calculate the geographical location coordinates of pedestrians.
Accurate geographical location capture and motion trajectory recording of pedestrians is realized, and no pedestrians need to carry any device, and can actively capture information of everyone in the area and upload and store big data in combination with surrounding geographical location information.
Smart Images

Figure CN110398760B_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to the field of analysis and monitoring manufacturing technology, and in particular to a pedestrian coordinate capture device based on image analysis and a use method thereof. Background Art
[0002] When pedestrians walk on the road, they do not necessarily carry a Beidou or GPS device. Even if they do, they may not actively connect to the platform to report their personal location information. If you want to obtain the geographic location of the target pedestrian, the device needs to actively discover the pedestrian, obtain relevant information about the pedestrian, and further calculate the pedestrian's geographic location information.
[0003] Defects in existing products and technology fields: There is currently no dedicated sensor device that actively captures the geographic location information of pedestrians. The current equipment that can be used for pedestrian location capture mainly uses image analysis technology to detect whether there are pedestrians within the monitoring range of the equipment. If so, the location information of the equipment itself is uploaded instead of the location information of the pedestrians. All targets that appear within the range of the equipment share a geographic location coordinate. This method itself does not actually calculate the location coordinates of the pedestrians, and the error is large. It cannot truly reflect the trajectory of the pedestrians, so the results cannot be guaranteed. In response to the above problems, a pedestrian coordinate capture device based on image analysis is designed. Summary of the invention
[0004] The purpose of the present invention is to provide a pedestrian coordinate capture device based on image analysis. After the capture device is started, it can actively capture the geographic location information, movement trajectory, and facial features of each person appearing in the area. Combined with the advantages of calibrated geographic location information of surrounding buildings, intersections, etc., it solves the problem that targets within the range of the device share a geographic location coordinate. This method itself does not actually calculate the position coordinates of pedestrians, has large errors, and cannot truly reflect the trajectory of pedestrians.
[0005] To achieve the above-mentioned purpose, the present invention adopts the following technical scheme: a pedestrian coordinate capture device based on image analysis, including a moving object video capturer, a processor, a pan-tilt servo mechanism, a distance capturer, and a close-up capturer. The video data collected by the moving object video capturer is sent to the processor, and the processor analyzes the pixel coordinates of the moving object from the received video data. The processor drives the pan-tilt servo mechanism according to the pixel coordinates to make the distance capturer aim at the moving object, and at the same time the close-up capturer captures close-up pictures of the moving object.
[0006] Furthermore, it also includes a coordinate collector for collecting the position coordinates of the capture device.
[0007] Furthermore, the distance collector is a laser rangefinder, which is used to collect the straight-line distance between the object and the capture device.
[0008] Furthermore, the laser rangefinder is arranged in parallel with a moving object video capturer, and the moving object video capturer is a video capturer used to capture videos around the capture device.
[0009] Furthermore, the video collector is a wide-angle camera or a 360-degree camera.
[0010] Furthermore, the close-up capturer is a camera for capturing close-up photos.
[0011] Furthermore, the laser rangefinder and the camera are fixed on the rotating part of the pan-tilt servo mechanism, and the pan-tilt servo mechanism is provided with an electronic compass for collecting the rotation angle and pitch angle of the pan-tilt servo mechanism.
[0012] A method for using a pedestrian coordinate capture device based on image analysis:
[0013] Step 1: Before the device works, the coordinate collector records the longitude and latitude coordinates of the device through the Beidou / GPS device<BD,LD> , the laser rangefinder records the device height h, which is initialized when the device is powered on;
[0014] Step 2: The wide-angle camera collects wide-angle video in the area, and the processor decomposes the video into each frame. It uses image analysis technology to detect moving objects or pedestrians in the picture and record the pixel coordinates of the moving objects or pedestrians.<x,y> ;
[0015] Step 3: Calculate the pixel coordinates of pedestrians in the wide-angle image<x,y> Converted to the attitude angle aligned with the gimbal servo mechanism<P,T> ; Wide-angle camera pixel coordinates<x,y> Attitude angle aligned with servo mechanism<P,T> The functional correspondence between them needs to be calibrated when the equipment is installed; by manually collecting a number of pixel coordinates and attitude angles, according to the fitting function model, the least squares fitting formula is used to obtain the correspondence between the two:
[0016] The fitting function model is as follows
[0017] P = a1*x+a2;
[0018] T = b1*x+b2*y+b3;
[0019] During the linkage process, enter the target pixel coordinates<x,y> , according to the fitting function, the attitude angle is directly calculated<P,T> ;
[0020] Step 4: The pan-tilt servo mechanism controls the stepper motor to aim the device at the target pedestrian according to the pixel coordinates of the pedestrian, and the close-up camera zooms in on the pedestrian accordingly; in step 2, the pixel coordinates of the pedestrian in the camera image coordinate system are obtained through image analysis technology, and in step 3, the pixel coordinates are converted into attitude angle coordinates. In step 4, multiple sensors are aimed at the target pedestrian through the servo linkage mechanism, which serves as the basis for further analysis of the pedestrian;
[0021] Step 5: After the device is aimed at the pedestrian, the laser rangefinder records the distance d between the pedestrian and the device, the electronic compass records the azimuth a of the aiming, and the close-up camera captures a close-up of the pedestrian. After the device is aimed at the pedestrian, this step can use multiple sensor fusion to analyze the relevant data of the pedestrian.
[0022] Step 6: According to the device's geographic location coordinates<BD,LD> , target distance d, azimuth angle a, calculate the target geographic location coordinates<B,L> The monitoring range of the device (e.g., within a radius of 100 meters) is relatively small. The surface of the earth where pedestrians in the area are located is simplified into a plane. First, the projection of the distance between the device and the target on the ground plane is calculated.
[0023]
[0024] db=dt*cos a
[0025] dl=dt*sina
[0026] B=BD+f(db,BD,LD)
[0027] L=LD+g(dl, BD, LD)f, g are obtained by machine learning data fitting. Before obtaining the formula, the device is installed in different scenes and aimed at different people. People use Beidou / GPS equipment to obtain geographic location coordinate samples. After collecting 3000 data samples, the quasi-Newton method is used to fit multiple function models, and finally the optimal function formula model is obtained;
[0028] Step 7: The close-up camera collects close-up video, decomposes the video into each frame, and extracts facial features in the picture;
[0029] Step 8: Repeat the above steps to assign an ID to each pedestrian. The geographic location coordinates of the target in each frame in the ID are combined to form the pedestrian's motion trajectory.
[0030] Step 9: Upload the movement trajectory, facial features, and close-up pictures of each pedestrian to the platform server via FTP.
[0031] Compared with the prior art, the present invention has the following beneficial effects:
[0032] 1. Image analysis technology is integrated with multiple sensors to collaborate with each other to capture pedestrians, calculate their geographic coordinates, and record their movement trajectories without requiring the target pedestrians to carry any devices;
[0033] 2. After the device is started, it can actively capture the geographic location information, movement trajectory, and facial features of every person in the area. Combined with the calibrated geographic location information of surrounding buildings, intersections, etc., it can analyze the regular activity areas and routes of each pedestrian and upload them as big data for storage. BRIEF DESCRIPTION OF THE DRAWINGS
[0034] The accompanying drawings are used to provide further understanding of the present invention and constitute a part of the specification. They are used to explain the present invention together with the embodiments of the present invention and do not constitute a limitation of the present invention.
[0035] In the attached picture:
[0036] Figure 1 A schematic diagram of the working process of a pedestrian coordinate capture device based on image analysis proposed by the present invention. DETAILED DESCRIPTION
[0037] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.
[0038] Reference Figure 1 A pedestrian coordinate capture device based on image analysis includes a moving object video capturer, a processor, a pan-tilt servo mechanism, a distance capturer, and a close-up capturer. The video data collected by the moving object video capturer is sent to the processor. The processor analyzes the pixel coordinates of the moving object from the received video data. The processor drives the pan-tilt servo mechanism according to the pixel coordinates to make the distance capturer aim at the moving object, and the close-up capturer collects close-up pictures of the moving object.
[0039] The pedestrian coordinate capture device based on image analysis also includes a coordinate collector for collecting the position coordinates of the capture device; the distance collector is a laser rangefinder, which is used to collect the straight-line distance between the object and the capture device; the laser rangefinder is arranged in parallel with the moving object video capturer, and the moving object video capturer is a video collector for collecting videos around the capture device; the video capturer is a wide-angle camera or a 360-degree camera; the close-up capturer is a camera for collecting close-up photos; the laser rangefinder and the camera are fixed on the rotating part of the pan-tilt servo mechanism, and the pan-tilt servo mechanism is provided with an electronic compass for collecting the rotation angle and pitch angle of the pan-tilt servo mechanism.
[0040] A method for using a pedestrian coordinate capture device based on image analysis:
[0041] Step 1: Before the device works, record the latitude and longitude coordinates of the device through the Beidou / GPS device<BD,LD> , the laser rangefinder records the device height h, which is initialized when the device is powered on;
[0042] Step 2: The wide-angle camera collects wide-angle video in the area, and the processor decomposes the video into each frame. It uses image analysis technology to detect moving objects or pedestrians in the picture and record the pixel coordinates of the moving objects or pedestrians.<x,y> ,This coordinate is the pixel coordinate in the wide-angle camera coordinate system, not the geographic coordinate of the pedestrian in the real world. To obtain the real geographic coordinate, it is necessary to further obtain the relative position relationship between the pedestrian and the device;
[0043] Step 3: Calculate the pixel coordinates of pedestrians in the wide-angle image<x,y> Converted to the attitude angle aligned with the gimbal servo mechanism<P,T> ; Wide-angle camera pixel coordinates<x,y> Attitude angle aligned with servo mechanism<P,T> The functional correspondence between them needs to be calibrated when the equipment is installed; by manually collecting a number of pixel coordinates and attitude angles, according to the fitting function model, the least squares fitting formula is used to obtain the correspondence between the two:
[0044] The fitting function model is as follows
[0045] P = a1*x+a2;
[0046] T = b1*x+b2*y+b3;
[0047] During the linkage process, enter the target pixel coordinates<x,y> , according to the fitting function, the attitude angle is directly calculated<P,T> ;
[0048] Step 4: The pan-tilt servo mechanism controls the stepper motor to aim the device at the target pedestrian according to the pixel coordinates of the pedestrian, and the close-up camera zooms in on the pedestrian accordingly; in step 2, the pixel coordinates of the pedestrian in the camera image coordinate system are obtained through image analysis technology, and in step 3, the pixel coordinates are converted into attitude angle coordinates. In step 4, multiple sensors are aimed at the target pedestrian through the servo linkage mechanism, which serves as the basis for further analysis of the pedestrian; this is an active discovery process, referred to as capture. In this way, no matter whether the pedestrian carries relevant equipment or not, as long as he appears in the field of view of the wide-angle camera, the pedestrian can be captured;
[0049] Step 5: After the device is aimed at the pedestrian, the laser rangefinder records the distance d between the pedestrian and the device, the electronic compass records the azimuth a of the aiming, and the close-up camera captures a close-up of the pedestrian. After the device is aimed at the pedestrian, this step can use multiple sensor fusion to analyze the relevant data of the pedestrian.
[0050] Step 6: According to the device's geographic location coordinates<BD,LD> , target distance d, azimuth angle a, calculate the target geographic location coordinates<B,L> The monitoring range of the device (e.g., within a radius of 100 meters) is relatively small. The surface of the earth where pedestrians in the area are located is simplified into a plane. First, the projection of the distance between the device and the target on the ground plane is calculated.
[0051]
[0052] db=dt*cosa
[0053] dl=dt*sina
[0054] B=BD+f(db,BD,LD)
[0055] L=LD+g(dl,BD,LD)
[0056] The two function formulas of f and g are obtained by machine learning data fitting. Before obtaining the formula, the device is installed in different scenes and aimed at different people. People use Beidou / GPS surveying equipment to obtain geographic location coordinate samples. After collecting 3,000 data samples, the quasi-Newton method is used to fit multiple function models, and finally the optimal function formula model is obtained;
[0057] Step 7: The close-up camera collects close-up video, decomposes the video into each frame, and extracts facial features in the picture;
[0058] Step 8: Repeat the above steps to assign an ID to each pedestrian. The geographic location coordinates of the target in each frame in the ID are combined to form the pedestrian's motion trajectory.
[0059] Step 9: Upload the movement trajectory, facial features, and close-up pictures of each pedestrian to the platform server via FTP.
[0060] The matters not described in detail in the present invention are all known technologies to those skilled in the art.
[0061] Although embodiments of the present invention have been shown and described, it will be appreciated by those skilled in the art that various changes, modifications, substitutions and variations may be made to the embodiments without departing from the principles and spirit of the present invention, and that the scope of the present invention is defined by the appended claims and their equivalents.
Claims
1. A method for using a pedestrian coordinate capture device based on image analysis, characterized in that: The steps include: Step 1: Before the device works, the coordinate collector records the longitude and latitude coordinates of the device through the Beidou / GPS device<BD,LD> , the laser rangefinder records the device height h, which is initialized when the device is powered on; Step 2: The wide-angle camera collects wide-angle video in the area, and the processor decomposes the video into each frame. It uses image analysis technology to detect moving objects or pedestrians in the picture and record the pixel coordinates of the moving objects or pedestrians.<x,y> ,This coordinate is the pixel coordinate in the wide-angle camera coordinate system, not the geographic coordinate of the pedestrian in the real world. To obtain the real geographic coordinate, it is also necessary to obtain the relative position relationship between the pedestrian and the device; Step 3: Calculate the pixel coordinates of pedestrians in the wide-angle image<x,y> Converted to the attitude angle aligned with the gimbal servo mechanism<P,T> ; Wide-angle camera pixel coordinates<x,y> Attitude angle aligned with the servo mechanism<P,T> The functional correspondence between them needs to be calibrated when the equipment is installed; by manually collecting a number of pixel coordinates and attitude angles, according to the fitting function model, the least squares fitting formula is used to obtain the corresponding relationship between the two: The fitting function model is as follows P = a1*x+a2; T = b1*x+b2*y+b3; During the linkage process, enter the target pixel coordinates<x,y> , according to the fitting function, the attitude angle is directly calculated<P,T> ; Step 4: The pan-tilt servo mechanism controls the stepper motor to aim the device at the target pedestrian according to the pixel coordinates of the pedestrian, and the close-up camera zooms in on the pedestrian accordingly; in step 2, the pixel coordinates of the pedestrian in the camera image coordinate system are obtained through image analysis technology, and in step 3, the pixel coordinates are converted into attitude angle coordinates. In step 4, multiple sensors are aimed at the target pedestrian through the servo linkage mechanism, which serves as the basis for analyzing the pedestrian; This is an active discovery process, referred to as capture. In this way, no matter whether the pedestrian carries relevant equipment or not, as long as he appears in the field of view of the wide-angle camera, he can be captured. Step 5: After the device is aimed at the pedestrian, the laser rangefinder records the distance d between the pedestrian and the device, the electronic compass records the azimuth a of the aiming, and the close-up camera captures a close-up of the pedestrian. After the device is aimed at the pedestrian, this step uses multiple sensor fusion to analyze the relevant data of the pedestrian. Step 6: According to the device's geographic location coordinates<BD,LD> , distance d, azimuth a, calculate the target geographic location coordinates<B,L> ; The monitoring range of the device is small, and the surface of the earth where the pedestrians in the area are located is simplified into a plane. First, the projection of the distance between the device and the target on the ground plane is calculated; db=dt*cosa dl=dt*sina B=BD+f(db,BD,LD) L=LD+g(dl,BD,LD) The two function formulas of f and g are obtained by machine learning data fitting. Before obtaining the formula, the device is installed in different scenes and aimed at different people. People use Beidou / GPS surveying equipment to obtain geographic location coordinate samples. After collecting 3,000 data samples, the quasi-Newton method is used to fit multiple function models, and finally the optimal function formula model is obtained; Step 7: The close-up camera collects close-up video, decomposes the video into each frame, and extracts facial features in the picture; Step 8: Repeat the above steps to assign an ID to each pedestrian. The geographic location coordinates of the target in each frame in the ID are combined to form the pedestrian's motion trajectory. Step 9: Upload the movement trajectory, facial features, and close-up pictures of each pedestrian to the platform server via FTP; The invention also includes a moving object video capturer, a processor, a pan-tilt servo mechanism, a distance capturer, a close-up capturer, and a collection and capture device. The video data collected by the moving object video capturer is sent to the processor. The processor analyzes the pixel coordinates of the moving object from the received video data. The processor drives the pan-tilt servo mechanism according to the pixel coordinates to make the distance capturer aim at the moving object, and the close-up capturer collects close-up pictures of the moving object. A coordinate collector for collecting the position coordinates of the capture device; The distance collector is a laser rangefinder, which is used to collect the straight-line distance between the object and the capture device. The laser rangefinder is arranged in parallel with the moving object video capturer, which is a video collector used to collect videos around the capture device, and the video collector is a wide-angle camera; A close-up capturer is a camera used to capture close-up photos.
2. The method for using the pedestrian coordinate capture device based on image analysis according to claim 1, characterized in that: The laser rangefinder and the camera are fixed on the rotating part of the pan-tilt servo mechanism, and the pan-tilt servo mechanism is provided with an electronic compass for collecting the rotation angle and pitch angle of the pan-tilt servo mechanism.
Citation Information
Patent Citations
Video monitoring device and tracking and recording method based on linkage camera
CN101547344A
Registration of 3d point cloud data to 2d electro-optical image data
CN102317972A
Unmanned aerial vehicle indoor positioning system and method
CN106441278A
A multi-camera data fusion method based on a spatial coordinate system
CN109190508A
Calculation method for automatic locating angle of pan-tilt-zoom camera
WO2014043973A1
