Motion track redisk method for fusion application of video track and GPS positioning

Abstract

The invention discloses an action track redisk method for fusion application of a video track and GPS positioning, and the method comprises the steps: obtaining the GPS positioning information of an actor in an action process, carrying out the denoising preprocessing of the positioning information, and forming a GPS track; calling available monitoring equipment position points in a certain range in the GPS action track, acquiring static factors of the monitoring equipment, collecting video monitoring pictures of action time periods, retrieving and identifying actions in the video monitoring pictures, determining longitude and latitude, height and time of the position of an actor, and serially connecting the longitude and latitude, height and time into a video track according to a time sequence; and taking the GPS track as a low-credibility track and the video track as a high-credibility track, and carrying out track deviation correction fusion to form an action replay track. According to the invention, GPS positioning deviation correction is carried out by using the video trajectory, blind compensation is carried out on the video trajectory by using GPS positioning, accurate and coherent recording of indoor and outdoor actions is realized, and the problem of difficult trajectory positioning in a complex action scene is solved.

Description

Technical Field

[0001] The present invention belongs to the technical field of action trajectory replay or backtracking, and in particular to an action trajectory replay method for integrating video trajectory with GPS positioning. Background Art

[0002] The core of action trajectory review is to accurately record each space-time trajectory. The traditional method uses a single GPS technology for positioning. In the process of reconnaissance or capture, the use of GPS equipment for positioning and tracking has become more common. This positioning method can achieve accurate positioning in large-scale spaces of more than 10 meters in wide outdoor environments, but the positioning accuracy is insufficient in indoor scenes such as urban commercial buildings, street community campuses, etc. First, because the GPS signal is shielded by buildings, it is difficult to penetrate multiple layers of walls to reach indoors; second, the plane positioning accuracy is not high enough to locate the exact position in a small indoor place; third, the height deviation is large, the height error is 2-3 times the plane distance error, and the height value jumps greatly.

[0003] In order to make up for the insufficient accuracy of GPS indoor positioning, WiFi positioning and Bluetooth positioning technologies have also been applied. The application premise of such technologies is relatively harsh, requiring special base stations to be deployed indoors and the person being tracked to wear corresponding beacon positioning devices. Currently, they are only used in a few special places such as smart manufacturing factories and prisons, and are not universally applicable.

[0004] With the popularization of video surveillance systems, most indoor scenes in cities, such as commercial complexes, entertainment venues such as bars / KTVs, smart parks, safe communities, and hotels, are equipped with video surveillance equipment to record the time and location of people appearing in the surveillance screen. Multiple surveillance devices can form a video trajectory by combining the order in which people appear. However, since the location of the surveillance equipment is fixed, the video trajectory has high accuracy; due to the small coverage range of the field of view, the video trajectory often has poor continuity. Once out of the field of view, the trajectory of its movement cannot be measured. Summary of the invention

[0005] The purpose of the present invention is to provide an action trajectory review method by integrating video trajectory and GPS positioning application to address the problems existing in the above-mentioned prior art.

[0006] The technical solution to achieve the purpose of the present invention is: a method for replaying an action trajectory by integrating video trajectory with GPS positioning, the method comprising:

[0007] Step 1: Obtain the GPS positioning information of the actor during the action, and perform denoising preprocessing on the positioning information to form a GPS trajectory;

[0008] Step 2, retrieve the available monitoring equipment location points within a certain range in the GPS action trajectory, obtain the static factors of the monitoring equipment, collect the video monitoring screens of the action period, and search and identify the action in the video monitoring screens, and at the same time determine the latitude and longitude, altitude and time of the actor's location, and connect them into a video trajectory in time sequence; the static factors include: the latitude and longitude, altitude, field of view and time offset of the monitoring equipment;

[0009] Step 3: Use the GPS trajectory as a low-credibility trajectory and the video trajectory as a high-credibility trajectory to perform trajectory correction and fusion to form an action review trajectory.

[0010] Furthermore, the video surveillance images collected during the action period in step 2 specifically include:

[0011] Simplify the GPS trajectory into a plane coordinate set P(x,y):

[0012] P(x,y) = {p0,p1,...,pm}

[0013] Where x is longitude, y is latitude, m is the number of segments, and pm is the mth segment trajectory;

[0014] For each monitoring device, calculate its distance to each trajectory and take the minimum distance. If the minimum distance is less than the preset distance, collect the video monitoring screen of the monitoring device during the action period.

[0015] Furthermore, for each monitoring device, calculating the distance to each track segment specifically includes:

[0016] Set up monitoring equipment The location is , Monitoring equipment The longitude and latitude of

[0017] (1) Take two trajectory points on the mth trajectory , Track points The longitude and latitude of Track points The longitude and latitude of

[0018] (2) Calculate trajectory vector , for:

[0019]

[0020]

[0021] (3) Determine whether the projection of the monitoring device in the trajectory direction is on the trajectory segment;

[0022] Calculation:

[0023]

[0024] If f < 0, return the monitoring device The actual distance to the m-th segment of the trajectory is:

[0025]

[0026] If f > 0, calculate the reference distance :

[0027]

[0028] If f > d, the actual distance is:

[0029]

[0030] If f < d, calculate the obtained projection point is:

[0031]

[0032] After that, calculate the actual distance is:

[0033]

[0034] In the formula, are respectively the longitude and latitude of the projection point

[0035] Further, the retrieving and identifying actions in the video monitoring screen in step 2 specifically include:

[0036] Perform pedestrian detection on the video monitoring screen. If a pedestrian is detected, extract the feature set of the pedestrian frame by frame, including several features;

[0037] Compare the feature set of the pedestrian with the action library. If all the features in the feature set do not match a certain action in the action library, it is determined that the pedestrian has no action and is discarded; if one or several features match a certain action in the action library, retain the corresponding video trajectory points and record the positions corresponding to the video trajectory points; if several features match different actions, according to the preset feature priority order, use the matching result corresponding to the feature with the highest priority as the final recognition result.

[0038] Further, the feature set includes one or more of face features, face attribute features, and body posture features.​

[0039] Furthermore, in step 2, the latitude and longitude, altitude, and time of the action location are determined by means of field of view segmentation and positioning, specifically: the field of view of the monitoring device is segmented to determine the specific location of the foot in the image, and then the location of the actor in the video is calculated according to the height, field of view angle, and GPS location of the monitoring device, specifically including:

[0040] Set up monitoring equipment The location is , the height is H, the pitch angle of the viewport center is A, and the direction angle of the monitoring device is B, where Monitoring equipment latitude and longitude; monitoring equipment The pixels are h×w;

[0041] (1) Calculate the distance between the actor and the monitoring device Distance for:

[0042]

[0043] In the formula, Indicates monitoring equipment Monitoring angle, is the vertical coordinate of the actor in the video frame;

[0044] (2) Calculating the actor relative to the monitoring device The deviation angle of the visual field centerline for:

[0045]

[0046] (3) Taking the monitoring device as the polar coordinate center and due north as the positive direction of the polar coordinate, the polar coordinate of the actor relative to the monitoring device is expressed as ( , );

[0047] (4) Obtain the position of the actor in the video by converting polar coordinates to rectangular coordinates .

[0048] Furthermore, step 3 specifically includes:

[0049] Combine GPS tracks and video tracks into the same track set in chronological order;

[0050] Traverse all points in the trajectory set to determine whether there is signal loss, position drift, or position conflict. If so, make corresponding trajectory corrections to obtain the action review trajectory.

[0051] Furthermore, in step 3, it is determined whether there is signal loss in the point trace, and the trajectory is corrected, which specifically includes:

[0052] Determine whether the time interval between two consecutive high-precision trajectory points exceeds a first preset threshold and does not exceed a second preset threshold. If so, it indicates that there is a short-term signal loss; the high-precision trajectory point refers to a point where the plane accuracy and height accuracy values ​​of the GPS point are within a preset range;

[0053] The direction and speed of the continuous high-precision trajectory points before the loss are extrapolated to estimate the travel path; specifically:

[0054] Take two consecutive high-precision trajectory points before the loss and , the interval between two points , estimate the nth trajectory point for:

[0055] .

[0056] Furthermore, in step 3, it is determined whether there is position drift in the point trace, and the trajectory is corrected, which specifically includes:

[0057] Determine whether the distance between the two track points exceeds the third preset threshold, and whether the track time interval does not exceed the fourth preset threshold. If both are yes, it indicates that there is position drift;

[0058] Remove the points with poor continuity between the two trajectory points, specifically:

[0059] Assume that the two trajectory points before and after are , ;

[0060] (1) Calculate the discontinuity of the two trajectory points respectively:

[0061]

[0062] In the formula, for The discontinuity value corresponding to the point, for The adjacent point before the point;

[0063] (2) For the two trajectory points, the trajectory point with a relatively large discontinuity value is removed.

[0064] Furthermore, in step 3, it is determined whether there is a position conflict in the point trace, and the trajectory is corrected, which specifically includes:

[0065] Determine whether the distance between the GPS track and the video position appearing at the same time is greater than a fifth preset threshold, if so, it indicates that there is a position conflict;

[0066] The GPS track is smoothed and corrected using continuous video track points, and the GPS track points between continuous video tracks are deleted, where the continuous video track points refer to the time difference between two video track points not exceeding the sixth threshold; specifically:

[0067] Assume the starting point of the continuous video trajectory is , the previous GPS track point is , then insert a smooth trajectory point between the two points for:

[0068] .

[0069] Compared with the prior art, the present invention has the following significant advantages:

[0070] (1) The present invention uses video trajectory to correct GPS positioning, uses GPS positioning to fill in the blind spots of video trajectory, and uses low-precision and high-precision trajectory points to achieve information complementarity, thereby achieving accurate and coherent recording of indoor and outdoor actions, and solving the problem of difficult trajectory positioning in complex indoor and outdoor action scenes.

[0071] (2) In response to abnormal situations such as position conflicts, trajectory drift, and short-term signal loss that occur during use, innovative trajectory correction methods such as a smoothing correction algorithm for video trajectory points to GPS trajectories and a drift point trajectory extrapolation estimation algorithm are proposed. This achieves the goal of accurately locating the trajectory of the personnel in action, with the trajectory accuracy controlled within 50m. The trajectory density can also be improved to form a stable and continuous trajectory, providing strong support for action review.

[0072] (3) The solution combines two positioning methods to achieve low-cost, high-precision and high-density mobile positioning. The method used can be implemented using a camera and a GPS chip. No special equipment or special positioning devices are required, and the dependence on equipment is low. This method has a wide range of applications and strong universality.

[0073] The present invention is further described in detail below in conjunction with the accompanying drawings. BRIEF DESCRIPTION OF THE DRAWINGS

[0074] Figure 1 The figure is a flow chart of a trajectory review method in one embodiment.

[0075] Figure 2 The figure is a feature matching flow chart in one embodiment.

[0076] Figure 3 A schematic diagram of video segmentation and trajectory calculation in one embodiment.

[0077] Figure 4 The figure is a flow chart of trajectory correction fusion in one embodiment. DETAILED DESCRIPTION

[0078] In order to make the purpose, technical solution and advantages of the present application more clearly understood, the present application is further described in detail below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present application and are not used to limit the present application.

[0079] It should be noted that if the embodiments of the present invention involve directional indications (such as up, down, left, right, front, back...), the directional indications are only used to explain the relative position relationship, movement status, etc. between the components under a certain specific posture (as shown in the accompanying drawings). If the specific posture changes, the directional indication will also change accordingly.

[0080] In addition, if there are descriptions involving "first", "second", etc. in the embodiments of the present invention, the descriptions of "first", "second", etc. are only used for descriptive purposes and cannot be understood as indicating or suggesting their relative importance or implicitly indicating the number of the indicated technical features. Therefore, the features defined as "first" and "second" may explicitly or implicitly include at least one of the features. In addition, the technical solutions between the various embodiments can be combined with each other, but they must be based on the ability of ordinary technicians in the field to implement them. When the combination of technical solutions is contradictory or cannot be implemented, it should be deemed that such a combination of technical solutions does not exist and is not within the scope of protection required by the present invention.

[0081] In one embodiment, in combination Figure 1 , provides an action trajectory review method for integrating video trajectory and GPS positioning, the method comprising:

[0082] Step 1: Obtain the GPS positioning information of the actor during the action, and perform denoising preprocessing on the positioning information to form a GPS trajectory;

[0083] Step 2, retrieve the available monitoring equipment location points within a certain range in the GPS action trajectory, obtain the static factors of the monitoring equipment, collect the video monitoring screens of the action period, and search and identify the action in the video monitoring screens, and at the same time determine the latitude and longitude, altitude and time of the actor's location, and connect them into a video trajectory in time sequence; the static factors include: the latitude and longitude, altitude, field of view and time offset of the monitoring equipment;

[0084] Step 3: Use the GPS trajectory as a low-credibility trajectory and the video trajectory as a high-credibility trajectory to perform trajectory correction fusion to obtain a continuous and accurate action review trajectory.

[0085] Furthermore, in one embodiment, the step 2 of collecting video surveillance images during the action period specifically includes:

[0086] Simplify the GPS trajectory into a plane coordinate set P(x,y):

[0087] P(x,y) = {p0,p1,...,pm}

[0088] Where x is longitude, y is latitude, m is the number of segments, and pm is the mth segment trajectory;

[0089] For each monitoring device, calculate its distance to each trajectory and take the minimum distance. If the minimum distance is less than the preset distance, collect the video monitoring screen of the monitoring device during the action period.

[0090] Furthermore, in one embodiment, the step of calculating the distance from each monitoring device to each track segment specifically includes:

[0091] Set up monitoring equipment The location is , Monitoring equipment The longitude and latitude of

[0092] (1) Take two trajectory points on the mth trajectory , Track points The longitude and latitude of Track points The longitude and latitude of

[0093] (2) Calculate trajectory vector , for:

[0094]

[0095]

[0096] (3) Determine whether the projection of the monitoring device in the trajectory direction is on the trajectory segment;

[0097] calculate:

[0098]

[0099] If f<0, return to the monitoring device The actual distance to the mth track for:

[0100]

[0101] If f > 0, calculate the reference distance :

[0102]

[0103] If f > d, the actual distance is:

[0104]

[0105] If f < d, calculate the projection point as:

[0106]

[0107] Then calculate the actual distance as:

[0108]

[0109] In the formula, are respectively the longitude and latitude of the projection point .

[0110] Furthermore, in one of the embodiments, in combination with Figure 2 , the retrieval and recognition of actions in the video surveillance screen in step 2 specifically include:

[0111] Perform pedestrian detection on the video surveillance screen. If a pedestrian is detected, extract the feature set of the pedestrian frame by frame, including several features;

[0112] Compare the feature set of the pedestrian with the action library. If all features in the feature set do not match a certain action in the action library, it is determined that the pedestrian has no action and is discarded; if one or several features match a certain action in the action library, retain the corresponding video trajectory points and record the positions corresponding to the video trajectory points; if the actions matched by several features are different, according to the preset feature priority order, use the matching result corresponding to the feature with the highest priority as the final recognition result.

[0113] Preferably here, the feature set includes one or more of face features, face attribute features, and body posture features.

[0114] Preferably here, the priority order is: face features, face attribute features, body posture features.

[0115] Further, in one of the embodiments, in step 2, the latitude and longitude, altitude and time of the action location are determined by means of field of view segmentation and positioning, specifically: the field of view of the monitoring device is segmented to determine the specific location of the foot in the image, and then the location of the actor in the video is calculated according to the height, field of view angle and GPS location of the monitoring device; Figure 3 , specifically including:

[0116] Set up monitoring equipment The location is , the height is H, the pitch angle of the viewport center is A, and the direction angle of the monitoring device is B, where Monitoring equipment latitude and longitude; monitoring equipment The pixels are h×w;

[0117] (1) Calculate the distance between the actor and the monitoring device Distance for:

[0118]

[0119] In the formula, Indicates monitoring equipment Monitoring angle, is the vertical coordinate of the actor in the video frame;

[0120] Here, if If the value of is greater than 50, The value is 50.

[0121] (2) Calculating the actor relative to the monitoring device The deviation angle of the visual field centerline for:

[0122]

[0123] (3) Taking the monitoring device as the polar coordinate center and due north as the positive direction of the polar coordinate, the polar coordinate of the actor relative to the monitoring device is expressed as ( , );

[0124] (4) Obtain the position of the actor in the video by converting polar coordinates to rectangular coordinates .

[0125] Further, in one embodiment, in combination Figure 4 , step 3 specifically includes:

[0126] Combine GPS tracks and video tracks into the same track set in chronological order;

[0127] Traverse all points in the trajectory set to determine whether there is signal loss, position drift, or position conflict. If so, make corresponding trajectory corrections to obtain the action review trajectory.

[0128] Here, signal loss, position drift, and position conflict are judged in order.

[0129] Further, in one embodiment, in step 3, determining whether there is signal loss in the point trace and performing trajectory correction specifically includes:

[0130] Determine whether the time interval between two consecutive high-precision trajectory points exceeds a first preset threshold and does not exceed a second preset threshold. If so, it indicates that there is a short-term signal loss; the high-precision trajectory point refers to a point where the plane accuracy and height accuracy values ​​of the GPS point are within a preset range; preferably, the preset range is 30 meters;

[0131] The direction and speed of the continuous high-precision trajectory points before the loss are extrapolated to estimate the travel path; specifically:

[0132] Take two consecutive high-precision trajectory points before the loss and , the interval between two points , estimate the nth trajectory point for:

[0133] .

[0134] Here, preferably, the first preset threshold is 1 minute and the second preset threshold is 15 minutes.

[0135] Further, in one embodiment, in step 3, determining whether there is position drift of the point trace and performing trajectory correction specifically includes:

[0136] Determine whether the distance between the two track points exceeds the third preset threshold, and whether the track time interval does not exceed the fourth preset threshold. If both are yes, it indicates that there is position drift;

[0137] Remove the points with poor continuity between the two trajectory points, specifically:

[0138] Assume that the two trajectory points before and after are , ;

[0139] (1) Calculate the discontinuity of the two trajectory points respectively:

[0140]

[0141] In the formula, for The discontinuity value corresponding to the point, for The adjacent point before the point;

[0142] (2) For the two trajectory points, the trajectory point with a relatively large discontinuity value is removed.

[0143] Here, preferably, the third preset threshold is 1 kilometer and the second preset threshold is 1 minute.

[0144] Furthermore, in one embodiment, in step 3, determining whether there is a position conflict in the point trace and performing trajectory correction specifically includes:

[0145] Determine whether the distance between the GPS track and the video position appearing at the same time is greater than a fifth preset threshold, if so, it indicates that there is a position conflict;

[0146] The GPS track is smoothed and corrected using continuous video track points, and the GPS track points between continuous video tracks are deleted, where the continuous video track points refer to the time difference between two video track points not exceeding the sixth threshold; specifically:

[0147] Assume the starting point of the continuous video trajectory is , the previous GPS track point is , then insert a smooth trajectory point between the two points for:

[0148] .

[0149] Here, preferably, the fifth preset threshold is 1 kilometer.

[0150] In one embodiment, an action trajectory review system using a fusion application of video trajectory and GPS positioning is provided, the system comprising:

[0151] The first module is used to obtain the GPS positioning information of the actor during the action, and perform denoising preprocessing on the positioning information to form a GPS trajectory;

[0152] The second module is used to retrieve the available monitoring equipment location points within a certain range in the GPS action trajectory, obtain the static factors of the monitoring equipment, collect the video monitoring screens during the action period, and search and identify the action in the video monitoring screens, while determining the latitude and longitude, altitude and time of the actor's location, and connect them into a video trajectory in time sequence; the static factors include: the latitude and longitude, altitude, field of view and time offset of the monitoring equipment;

[0153] The third module is used to use the GPS trajectory as a low-credibility trajectory and the video trajectory as a high-credibility trajectory to perform trajectory correction and fusion to form an action review trajectory.

[0154] For the specific definition of the action trajectory review system for the fusion application of video trajectory and GPS positioning, please refer to the definition of the action trajectory review method for the fusion application of video trajectory and GPS positioning above, which will not be repeated here. Each module in the above-mentioned action trajectory review system for the fusion application of video trajectory and GPS positioning can be implemented in whole or in part through software, hardware and a combination thereof. The above-mentioned modules can be embedded in or independent of the processor in the computer device in the form of hardware, or can be stored in the memory of the computer device in the form of software, so that the processor can call and execute the operations corresponding to the above modules.

[0155] In one embodiment, a computer device is provided, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein when the processor executes the computer program, the following is achieved:

[0156] Step 1: Obtain the GPS positioning information of the actor during the action, and perform denoising preprocessing on the positioning information to form a GPS trajectory;

[0157] Step 2, retrieve the available monitoring equipment location points within a certain range in the GPS action trajectory, obtain the static factors of the monitoring equipment, collect the video monitoring screens of the action period, and search and identify the action in the video monitoring screens, and at the same time determine the latitude and longitude, altitude and time of the actor's location, and connect them into a video trajectory in time sequence; the static factors include: the latitude and longitude, altitude, field of view and time offset of the monitoring equipment;

[0158] Step 3: Use the GPS trajectory as a low-credibility trajectory and the video trajectory as a high-credibility trajectory to perform trajectory correction and fusion to form an action review trajectory.

[0159] For the specific limitations of each step, please refer to the limitations of the action trajectory review method for the fusion application of video trajectory and GPS positioning above, which will not be repeated here.

[0160] In one embodiment, a computer-readable storage medium is provided, on which a computer program is stored, and when the computer program is executed by a processor, the computer program implements:

[0161] Step 1: Obtain the GPS positioning information of the actor during the action, and perform denoising preprocessing on the positioning information to form a GPS trajectory;

[0162] Step 2, retrieve the available monitoring equipment location points within a certain range in the GPS action trajectory, obtain the static factors of the monitoring equipment, collect the video monitoring screens of the action period, and search and identify the action in the video monitoring screens, and at the same time determine the latitude and longitude, altitude and time of the actor's location, and connect them into a video trajectory in time sequence; the static factors include: the latitude and longitude, altitude, field of view and time offset of the monitoring equipment;

[0163] Step 3: Use the GPS trajectory as a low-credibility trajectory and the video trajectory as a high-credibility trajectory to perform trajectory correction and fusion to form an action review trajectory.

[0164] For the specific limitations of each step, please refer to the limitations of the action trajectory review method for the fusion application of video trajectory and GPS positioning above, which will not be repeated here.

[0165] As a specific example, the present invention is described in detail in one of the embodiments.

[0166] 1. Use portable positioning equipment to collect GPS positioning information during the action, perform denoising preprocessing on the positioning information, and form a GPS track;

[0167] 1) Use smart phones, portable positioning devices, beacons and other positioning devices to collect GPS positioning information, encapsulate the GPS information into a custom format message of [id, collection time, longitude, latitude, altitude, speed, azimuth, plane accuracy, altitude accuracy, speed accuracy, azimuth accuracy], and transmit it back through the message queue rabbitMQ;

[0168] 2) After receiving GPS information, the system reads the plane accuracy and altitude accuracy values ​​of the GPS point. Points with plane and altitude accuracy within 30 meters are marked as high-precision points, points with plane and altitude accuracy between 30-100 meters are recorded as low-precision points, and points with plane accuracy or altitude accuracy greater than 100 meters are discarded, thus forming the original GPS track (hereinafter referred to as GPS track).

[0169] 2. Retrieve the available monitoring equipment within 500 meters of the GPS action track, locate it from the device screen, and use the screen segmentation positioning algorithm to generate the video track;

[0170] 1) Simplify the GPS trajectory into a set of trajectory plane coordinate points , the camera position is recorded as , is the longitude, is the latitude, is the number of line segments, and the number of cameras within 500 meters of the GPS track is calculated. Assume that the camera position is , take the trajectory part Two points;

[0171] Calculate the trajectory vector , as

[0172]

[0173]

[0174] Judge whether the projection of the camera in the trajectory direction is on the trajectory segment:

[0175]

[0176] When f < 0, return the distance as:

[0177]

[0178] When f > 0, calculate the reference distance :

[0179]

[0180] If f > d, the distance is:

[0181]

[0182] If f < d, the projection point can be obtained as:

[0183]

[0184] Then calculate the actual distance as:

[0185]

[0186] Traverse all trajectory segments according to the above method, and obtain the minimum distance as the minimum distance between the camera and the trajectory.

[0187] 2) Use openCV to extract frames from the video, use labelme to annotate the pedestrian dataset, and use YOLOv3 to train and detect pedestrians in the video, where Darknet-53 is selected as the backbone network to extract pedestrian features.

[0188] 3) Use the open-source InsightFace project for face recognition. The dataset uses the Chinese Academy of Sciences Asian face dataset CAS-PEAL, and ResNet is used as the backbone network to extract the face features of pedestrians.

[0189] 4) FaceAttribute-FAN, an open source project of Tencent Youtu, is used for face attribute recognition. The network structure uses Facial abstraction net, and the backbone network uses resnet. The focus is on identifying the identity of the person, such as whether the person has a beard, fatness or thinness, gender, whether the person has makeup, the size of the glasses, whether the person has hair, whether the person wears a hat, and whether the person wears glasses.

[0190] 5) The Fast-ReID project is used for pedestrian body feature recognition. The Market-1501 dataset of Tsinghua University pedestrian re-identification dataset is used for training and testing. Random patch preprocessing software is used for data augmentation. ResNet is selected as the backbone network. Max pooling is used for global body sign aggregation. The training strategy is set to Learningrate. Kuangshi Circle loss is used as the loss function for model training.

[0191] 6) Using one or more of facial features, facial attribute features, and body features, gradually search and identify in the monitoring screen. The specific process is as follows: Figure 2 shown.

[0192] 7) After positioning, obtain the latitude and longitude of the camera device , camera height H, viewport center pitch angle A (A < 90°), camera orientation angle B, and use the field of view segmentation positioning algorithm to calculate the longitude and latitude of the location. Figure 2 As shown, the actor's position in the video The estimation method is as follows:

[0193] The horizontal distance from the center of the camera screen to the camera for:

[0194]

[0195] Assuming the camera monitoring angle is 2C (C < 45° and C < A), the closest shot , Maximum shooting distance for:

[0196]

[0197]

[0198] like >50, Take 50.

[0199] Take a 1K standard definition camera (1920*1080 resolution) and assume that the nearest edge of the video frame is =0, the farthest edge of the video frame =1080, the position of the person in the video frame is , then take the distance between the actor and the camera for,

[0200]

[0201] The deviation angle of the actor relative to the camera's field of view for:

[0202]

[0203] Therefore, the camera is taken as the center of the polar coordinates, and the north direction is the positive direction of the polar coordinates. The polar coordinates relative to the camera can be expressed as ( , ), through the formula of converting polar coordinates to rectangular coordinates, the position of the actor in the video can be obtained.

[0204] 3. Combine GPS tracks and video tracks into the same track set in chronological order , traverse all trajectory points at once, determine whether there are signal loss, position drift, or position conflict problems, and use corresponding processing methods to correct the trajectory to obtain a continuous and accurate action review trajectory .

[0205] 1) Short-term signal loss correction

[0206] If the time interval between two consecutive trajectory points is more than 1 minute but less than 15 minutes, it is defined as short-term signal loss and trajectory extrapolation is used for correction. Points There is a track loss, the interval time with the previous point , take the trajectory point before loss and , the interval between two points , calculate the nth trajectory point for:

[0207]

[0208] 2) Position drift correction

[0209] If the track distance between the two points exceeds 1 km and the track time interval does not exceed 30 seconds, it is defined as position drift, and the points with poor continuity between the front and rear points need to be removed for correction.

[0210] Assume that the three consecutive points are , , ,point Discontinuity for:

[0211]

[0212] Calculate the discontinuity between two points before and after , , retaining good continuity ( Small) points.

[0213] 3) Position conflict correction

[0214] If the distance between the GPS track and the video track points that appear alternately in the same time period is greater than 1 km, it is defined as a position conflict. The GPS track is smoothed using continuous video track points (the time difference between two video track points does not exceed 1 minute), and the GPS track points between continuous video tracks are deleted. Assume that the starting point of the continuous video track is , the previous GPS track point , then insert a smooth trajectory point between the two points for:

[0215] .

[0216] The above shows and describes the basic principles, main features and advantages of the present invention. Those skilled in the art should understand that the present invention is not limited by the above embodiments, and the above embodiments and descriptions are only for explaining the principles of the present invention. Without departing from the spirit and scope of the present invention, any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present invention should be included in the protection scope of the present invention.

Claims

1. A method for replaying an action trajectory by integrating video trajectory and GPS positioning, characterized in that: The method comprises: Step 1: Obtain the GPS positioning information of the actor during the action, and perform denoising preprocessing on the positioning information to form a GPS trajectory; Step 2, retrieve the available monitoring equipment location points within a certain range in the GPS action trajectory, obtain the static factors of the monitoring equipment, collect the video monitoring screens of the action period, and search and identify the action in the video monitoring screens, and at the same time determine the latitude and longitude, altitude and time of the actor's location, and connect them into a video trajectory in time sequence; the static factors include: the latitude and longitude, altitude, field of view and time offset of the monitoring equipment; Step 3: Use the GPS trajectory as a low-credibility trajectory and the video trajectory as a high-credibility trajectory to perform trajectory correction and fusion to form an action review trajectory.

2. The action trajectory review method of the video trajectory and GPS positioning fusion application according to claim 1 is characterized in that: The video surveillance footage collected during the action period in step 2 specifically includes: Simplify the GPS trajectory into a plane coordinate set P(x,y): P(x,y) = {p0,p1,...,pm}; Where x is longitude, y is latitude, m is the number of segments, and pm is the mth segment trajectory; For each monitoring device, calculate its distance to each trajectory and take the minimum distance. If the minimum distance is less than the preset distance, collect the video monitoring screen of the monitoring device during the action period.

3. The action trajectory review method of the video trajectory and GPS positioning fusion application according to claim 2 is characterized in that: The step of calculating the distance from each monitoring device to each track segment specifically includes: Set up monitoring equipment The location is , Monitoring equipment The longitude and latitude of (1) Take two trajectory points on the mth trajectory , Track points The longitude and latitude of Track points The longitude and latitude of (2) Calculate trajectory vector , for: ； ； (3) Determine whether the projection of the monitoring device in the trajectory direction is on the trajectory segment; calculate: ； If f<0, return to the monitoring device The actual distance to the mth track for: ； If f>0, calculate the reference distance : ； If f>d, the actual distance for: ； If f < d, the calculated projection point is as follows: ； Then calculate the actual distance for: ； In the formula, The projection points The longitude and latitude of .

4. The action trajectory review method of the video trajectory and GPS positioning fusion application according to claim 1 is characterized in that: The step 2 of searching and identifying actions in the video surveillance screen specifically includes: Perform pedestrian detection on the video surveillance screen. If a pedestrian is detected, extract the pedestrian's feature set frame by frame, including several features; The pedestrian's feature set is compared with the action library. If all the features in the feature set do not match an action in the action library, it is determined that the pedestrian has no action and is discarded; if one or several features match an action in the action library, the corresponding video trajectory points are retained and the corresponding positions of the video trajectory points are recorded; if the actions matched by several features are different, the matching result corresponding to the feature with the highest priority is taken as the final recognition result according to the preset feature priority order.

5. The action trajectory review method of the video trajectory and GPS positioning fusion application according to claim 4 is characterized in that: The feature set includes one or more of facial features, facial attribute features, and body features.

6. The action trajectory review method of the video trajectory and GPS positioning fusion application according to claim 1 is characterized in that: In step 2, the latitude and longitude, altitude, and time of the action location are determined by segmenting the field of view. Specifically, the field of view of the monitoring device is segmented to determine the specific location of the foot in the image. Then, the location of the actor in the video is calculated based on the height, field of view angle, and GPS location of the monitoring device. Specifically, it includes: Set up monitoring equipment The location is , the height is H, the pitch angle of the viewport center is A, and the direction angle of the monitoring device is B, where Monitoring equipment latitude and longitude; monitoring equipment The pixels are h×w; (1) Calculate the distance between the actor and the monitoring device Distance for: ； In the formula, Indicates monitoring equipment Monitoring angle, is the vertical coordinate of the actor in the video frame; (2) Calculating the actor relative to the monitoring device The deviation angle of the visual field centerline for: ； (3) Taking the monitoring device as the polar coordinate center and due north as the positive direction of the polar coordinate, the polar coordinate of the actor relative to the monitoring device is expressed as ( , ); (4) Obtain the position of the actor in the video by converting polar coordinates to rectangular coordinates .

7. The action trajectory review method of the video trajectory and GPS positioning fusion application according to claim 1 is characterized in that: Step 3 specifically includes: Combine GPS tracks and video tracks into the same track set in chronological order; Traverse all points in the trajectory set to determine whether there is signal loss, position drift, or position conflict. If so, make corresponding trajectory corrections to obtain the action review trajectory.

8. The method for replaying the action trajectory by integrating the video trajectory with GPS positioning according to claim 7 is characterized in that: In step 3, it is determined whether there is signal loss in the point trace and the trajectory is corrected, which includes: Determine whether the time interval between two consecutive high-precision trajectory points exceeds a first preset threshold and does not exceed a second preset threshold. If so, it indicates that there is a short-term signal loss; the high-precision trajectory point refers to a point where the plane accuracy and height accuracy values ​​of the GPS point are within a preset range; The direction and speed of the continuous high-precision trajectory points before the loss are extrapolated to estimate the travel path; specifically: Take two consecutive high-precision trajectory points before the loss and , the interval between two points , estimate the nth trajectory point for: 。 9. The method for replaying the action trajectory by integrating the video trajectory and GPS positioning according to claim 7 is characterized in that: In step 3, it is determined whether there is position drift in the point trace and the trajectory is corrected, which specifically includes: Determine whether the distance between the two track points exceeds the third preset threshold, and whether the track time interval does not exceed the fourth preset threshold. If both are yes, it indicates that there is position drift; Remove the points with poor continuity between the two trajectory points, specifically: Assume that the two trajectory points before and after are , ; (1) Calculate the discontinuity of the two trajectory points respectively: ； In the formula, for The discontinuity value corresponding to the point, for The adjacent point before the point; (2) For the two trajectory points, the trajectory point with a relatively large discontinuity value is removed.

10. The action trajectory review method of the video trajectory and GPS positioning fusion application according to claim 7 is characterized in that: In step 3, it is determined whether there is a position conflict in the point trace and the trajectory is corrected, which specifically includes: Determine whether the distance between the GPS track and the video position appearing at the same time is greater than a fifth preset threshold, if so, it indicates that there is a position conflict; The GPS track is smoothed and corrected using continuous video track points, and the GPS track points between continuous video tracks are deleted, where the continuous video track points refer to the time difference between two video track points not exceeding the sixth threshold; specifically: Assume the starting point of the continuous video trajectory is , the previous GPS track point is , then insert a smooth trajectory point between the two points for: 。

Images